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Abstract. We give a detailed description of the resolution of the identity of 
a second order g-difference operator considered as an unbounded self-adjoint 
operator on two different Hilbert spaces. The g-difference operator and the two 
choices of Hilbert spaces naturally arise from harmonic analysis on the quan- 
tum group SUq{l, 1) and SUq{2). The spectral analysis associated to SUq{l, 1) 
leads to the big q-Jacobi function transform together with its Plancherel mea- 
sure and inversion formula. The dual orthogonality relations give a one- 
parameter family of non-extremal orthogonality measures for the continuous 
dual (;~^-Hahn polynomials with > 1, and explicit sets of functions which 
complement these polynomials to orthogonal bases of the associated Hilbert 
spaces. The spectral analysis associated to SUq{2) leads to a functional ana- 
lytic proof of the orthogonality relations and quadratic norm evaluations for 
the big g-Jacobi polynomials. 



1. Introduction 

Harmonic analysis on the compact quantum group SUq{2) has been studied by 
several authors, see for example |2|], 0, Q. One possible approach is study- 
ing the restriction of the Haar functional and of the action of the quantum Casimir 
to "functions" in SUq{2) of a given, fixed bi-T-type, where T is the standard com- 
pact torus in SU(2). This reduces the harmonic analysis to the spectral analysis of 
an explicit second order g-difference operator considered as unbounded linear oper- 
ator on an explicit Hilbert space. The Hilbert space is the L^-space corresponding 
to the orthogonality measure of the little g-Jacobi polynomials, while the second 
order g-difference operator is diagonalized by the little g-Jacobi polynomials, so 
this leads to an interpretation of (the orthogonality relations of) the little (/-Jacobi 
polynomials on SUq{2). 

This approach was employed by Kakehi, Masuda and Ueno and Kakehi 
|Tl| for the development of harmonic analysis on the non-compact quantum group 
SUq{l^ 1), see also Vaksman and Korogodsky ||l5|. The restriction of the Casimir 
to functions on SUq{l, 1) of a given, fixed bi-T-type leads to the same second order 
g-difference operator as for SUq{2), while the restriction of a Haar functional on 
SUq{l,l) to the given bi-T-type leads to a i^-space with respect to a discrete 
measure with unbounded support of the form [O,oo(z))q = {zq*'}kez {z ^ 0), 
where < g < 1. The spectral analysis of the second order q-difference operator 
considered as unbounded linear operator on this Hilbert space was developed in 
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|T2[ , ||Tl[ . It leads to the Plancherel formula and the inversion formula for the little 
g-Jacobi function transform. 

It is well known that harmonic analysis on SUq{2) can be generalized by re- 
placing the role of the torus T in SUq{2) by a "conjugate" Tt (t g M), which is 
defined in terms of twisted primitive elements in the quantized universal envelop- 
ing algebra J7g(su(2)), see p^ . In particular, considering the harmonic analysis on 
SUq{2) with respect to left T- types and right Tt-types leads to an interpretation 



of the big q-Jacobi polynomials on SUq{2), see [19 . Furthermore, the restriction 
of the Haar functional to functions of a given left T-type and right T(-type can be 
computed directly and identified with the orthogonality measure of the big g- Jacobi 
polynomials, see [|l5| for the spherical case. 

In this paper we develop the functional analytic aspects related to the harmonic 
analyis on SUq{l, 1) with respect to left T-types and right T^-types. Concretely, we 
give a detailed description of the spectral properties of the second order q-difference 
operator L which is diagonalized by the big g-Jacobi polynomials, considered as an 
unbounded linear operator on a one-parameter family of Hilbert spaces. The Hilbert 
spaces are L^-spaces corresponding to explicit discrete measures with unbounded 
support [— l,oo(z))g = {— '?'^}feez+ U {zq^}k(^z {z > 0). They can be interpreted 
for specific parameter values as the restriction of a Haar functional on SUq(l, 1) to 
functions of a given left T-type and right T(-type. 

We construct a one-parameter family of dense domains for which L is self-adjoint. 
With respect to one of these domains, we explicitly compute the resolution of the 
identity of L. This leads to the big g-Jacobi function transform, its Plancherel for- 
mula and its inversion formula. Furthermore, we show that the corresponding dual 
orthogonality relations lead to a one-parameter family of non-extremal orthogonal- 
ity measures for the continuous dual g^-'^-Hahn polynomials, and explicit sets of 
functions complementing these polynomials to orthogonal bases of the correspond- 
ing Hilbert spaces. 

The little (respectively big) g-Jacobi function transform is a g-analogue of the 
Jacobi function transform on the interval [0, oo) (respectively [— l,cx))), see e.g. 
Braaksma and Meulenbeld Q, Koornwinder and references given there. In the 
classical setting, these two transforms are related by a dilation of the geometric 
parameter in which the end-point of the interval [0, oo) is mapped onto the end- 
point —1 of the interval [— l,oo). The key point in our study of the big g- Jacobi 
function transform is to show that the limiting point of the g-interval [—1, oo{z))q 
does not play a special role in the spectral analysis, while the role of the end-point 
— 1 is similar to the role of the limiting point for the little g- Jacobi function 
transform, see [pl]| . 

We will report elsewhere in detail about the connection between the big g- Jacobi 
function transform and harmonic analysis on SUq{l, 1). Furthermore, we will report 
elsewhere on the Plancherel formula and the inversion formula for the Askey- Wilson 
function transform, which is associated to harmonic analysis on SUq{l, 1) with 
respect to left T^-types and right Tt-types (s, t e R). 

The organization of the paper is as follows. 

In section 2 we introduce the second order g-difference operator L and the Hilbert 
spaces on which we consider L as an unbounded linear operator. Furthermore, we 
derive dense domains for which L is self-adjoint. The domains are given explicitly 
in terms of continuity and differentiability conditions of the functions at the origin. 
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In section 3 we derive criteria when it is possible to extend an arbitrary eigen- 
function of L on the positive real axis to a global eigenfunction in such a way that 
the solution is "continuously differentiable" at the origin. We furthermore give, in 
terms of basic hypergeometric series, two explicit eigenfunctions which are contin- 
uously differentiable at the origin. One of them is the spherical function for the big 
g-Jacobi function transform. 

In section 4 we introduce the asymptotically free eigenfunction of L on the pos- 
itive real axis and we give the corresponding c-function expansion of the spherical 
function on the positive real axis. 

In section 5 we extend for generic parameters the asymptotically free solution to a 
global eigenfunction of L in such a way, that the global eigenfunction is continuously 
differentiable at the origin. This leads to the c-function expansion of the spherical 
function on the whole support of the measure. 

In section 6 we define the Green function of L in terms of the spherical function 
and the extended asymptotically free eigenfunction. 

In section 7 we use the Green function to compute the continuous contribution to 
the resolution of the identity of L. Furthermore, we derive the Plancherel formula 
and the inversion formula for the continuous part of the big q- Jacobi function 
transform. 

In section 8 we derive the discrete contribution of the resolution of the identity 
of L and we derive the orthogonality relations and the quadratic norm evaluations 
of the corresponding spherical functions, which are square integrable in these cases. 
We furthermore give a precise description of the spectrum of L. 

In section 9 we state and prove the main results of the paper. We derive the 
Plancherel formula and the inversion formula for the big g- Jacobi function trans- 
form. Furthermore, we show that the dual orthogonality relations lead to a one- 
parameter family of non-extremal orthogonality measures for the continuous dual 
g~^-Hahn polynomials, as well as explicit sets of functions which complement the 
polynomials to orthogonal bases of the corresponding Hilbert spaces. 

Finally we derive in section 10 a functional analytic proof of the orthogonality 
relations and the quadratic norm evaluations for the big g- Jacobi polynomials. 
Proofs will only be sketched in this section, we mainly emphasize the differences 
between the compact setting, which corresponds to SUq{2) and the big g- Jacobi 
polynomials, and the non-compact setting, which corresponds to SUq{\, 1) and the 
big g-Jacobi functions. 

Notations: We assume throughout the paper that < g < 1 is fixed. We follow 
the notation of Gasper and Rahman concerning g-shifted factorials and basic 
hypergeometric series. We write 

9{xi, ... ,Xr) = 0{xi) . . .9{Xr) 

for products of (renormalized) Jacobi theta-products 9[x) — (x, q/x: g) . We write 
Z+ = {0, 1, 2, . . . , } and N = {1, 2, . . . }. 
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2. The second order (j-difference operator 

In this section we consider the second order q-difference operator L associated 
with the big g-Jacobi polynomials as an unbounded operator on a one-parameter 
family of Hilbert space. We determine suitable domains of definition for which L 
is self-adjoint. 

The second order g-difference operator L depends on three parameters (a,5, c). 
The corresponding one-parameter family of Hilbert spaces thus depends on four 
parameters (a, 5, c, z), where the parameter z labels the Hilbert spaces. In sections 
^|~^ we assume that the four parameters (a, 6, c, z) satisfy z > and (a, b, c) G V , 
where 

= {(o, 6, c) I a, 6, c > 0, ah,ac,hc<l}, (2.1) 

unless explicitly stated otherwise. Observe that two of the three parameters a, 6, c 
take their values in the open interval (0, 1). 
We define the g-interval / by 

/ = [-1, oo{z))q = {^q'' \keZ+}(J {zq'' | fc G Z}, 

which is regarded as a discrete g-analogue of the interval [— 1 , oo) . We use the 
notation (— l,oo(z))g for the g-interval / \ {—1} = [—q, oo{z))q. Let be the 

linear space of complex- valued functions / : / — > C, and define a linear operator 
LeEndc(.F(/)) by 

L = Ai-){Tg - Id) + B{■){Tg-^ - Id), (2.2) 
with the g-shift operators defined by (Tg±if){x) — f{q^^x), and with 



Here (L/)(— 1) is by definition given by 

{Lf){-l)^A{-l){f{-q)-f {-!)), feHl). (2.4) 

which is formally compatible with the fact that B{—\) — 0. 

In the next lemma we rewrite the operator L as a second order operator in the 
g-difference operator Dq, which is defined by 

Lemma 2.1. Let f E 3-{I), then 

{p{x)(DM-)D,l)){q-'i), I £ (-1, 
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where 

(~abx, —acx; q) 

pi^) = Vi- — 

—OCX, —qx; q) 

V X (2.6) 

. X (1-9)^ {~bcx,~qx;q} 

= T, 7 1 

qoc y—qabx, —qacx; qj ^ 

Proof. Observe that p and r are well defined as functions on the g-interval I 
since (a, 6, c) S V. The formula for x G (—1,00(2:))^ follows by observing that 
p{-)Tq-iDqr{-)Dq IS of the form ( p^ ) with A and B given by 

^ _ qp{x)r{x) ^ _ q^p{x)r{q-^x) 

{l-qfx^' il-q)^x^ ■ 

By inserting the explicit functions p and r, we see that the corresponding A and 
B coincide with the ones given by (2.3). The formula for Lf in the point x = — 1 
follows now immediately from the definition ( p.4[ ) of 1). □ 

We have that p{x) > for all a; G / since (a, b, c) G V. We can thus define the 
Hilbert space 

n = {fem I ll/f = (/,/)< 00}, (2.7) 

where 

(f^9)=J^ f{x)g{x)^y (2.8) 
with the Jackson g-integral for a, 7 G C* defined by 

f{x)dqX^ / f{x)dqX'- / f{x)dqX, P^J,Oo{j) 

I Jo Jo 

/(x)d,a: - (1 - <Z) ^ /(7g")7g", (2.9) 

n=0 

(7) 



/ /(x)d,a; = (1 - <z) ^ /(79")79", 
.^0 „ — — , 



n— — 00 



for functions / such that the sums converge absolutely. If a = (7*^7 for certain 
k G Z+, then the g-integral from a to 7 can be defined for arbitrary functions /, 
since the g-integral reduces then to a finite sum. 

We regard L as an unbounded operator on the Hilbert space Ti. In the remainder 
of this section, we define a one-parameter family of dense subspaces V C Ti such that 
L, with domain of definition T), is self-adjoint. In order to determine the subspaces, 
it is convenient to consider the symmetry of L with respect to a truncated version 
of the inner product (., .), which is defined for k G Z+ and l,m d with Z < m by 

{f,g)k;Lrn=[ I + I )/(:r)^-^ 




p{x) 
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Observe that for all /, g G ?i, we have 

hm (/,g)fc;i,™ = (/,5)- (2.10) 



l^ — oo 



We have now the fohowing lemma. 

Lemma 2.2. Let f, g G k £ Z+ and l^m G 1^ with I < m, then 



with f{x) — f{x) and with the Wronskian W{f,g) G defined by 

W{f,g){x) ^ -/(ga;).9(x)) 

= qr{x){{Dgf){x)g{x) - f{x){D,g){x)) 

for all X I . 



Proof. Since p and r (respectively A and B) are real valued on the g-interval /, we 
may restrict the proof to real valued functions / and g. Then we derive by a direct 
computation using Lemma |2.l| that 

{{Lf)ix)gix) ~ fi^){Lg)ix)]^-^^^ 

^iwif,g){q-^x)-Wif,g){x), x £ (-1, oo(z))„ 
[-W{f,g){-1), x = -l. 

The lemma is now an easy consequence of this formula since the finite sums become 
telescoping. □ 

In the remainder of this section, we use some standard terminology on unbounded 
linear operators, see Chapter XII] and |20, Chapter 13]. Let V cHhe a, dense 
linear space satisfying L(T>^ C H. Then V may be considered as a domain of 
definition for the unbounded operator L on Ti.. Lemma |2.2| and ( 2.10| ) then show 



that {L,V) is a densely defined symmetric operator if and only if for all f,g'E'D, 
the three limits 

hm lim T^(/,g)(z(z™), lim T^(/,g)(V) 

— oo m— »oo k—^QG 

exist, and 

hm W{f,g){^q'-')- lim W{f,g) (zq^) + lim = 0. 

The following lemma determines the behaviour of the Wronskian in the limit to 
infinity. 

Lemma 2.3. Let f,g ETC, then liTHm^oo W{f,g){zq^"^) = 0. 
Proof. Using the formula 

{xq'-^-q)^ = {-xrq-"^^'^-''^/\l/x-q)Jqx-,q)^, x e C* , (2.12) 
it follows that the behaviour of the weights of (., .) at infinity is given by 

i^-^^^ = (1 + 0(g")) , m^oo, (2.13) 
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where K is the positive constant 

(2.14) 

0[—abz, —acz) 

This imphes that Umm_,oo a~"^ f{zq^"^) = for aU f G Ti. The proof follows now 
from the definition of the Wronskian ( p.ll ) and the fact that 

'^'■^^'^""'^ = if a2-2™ (1 + 0(9™)) , m^oo. (2.15) 



(1 - q)zq- 

□ 



It follows from Lemma 2.3 that the domains V cTi for which (L, V) is symmetric 
are determined by vanishing properties of the Wronskian at the origin. Before 
we define suitable domains of definition for L explicitly, we first introduce some 
convenient notations. For / G J^{I), we define 

/(0+) = hm fizq''), /(0-) = hm f{-q''), 

/'(0+) = hm {Dj)izq''), no-) = hm {Dj)i-q'^) 

provided that the limits exist. From now on we tacitly assume that the limits exist 
whenever we write /(0+), /(O^) etc. Let T = {a G C | |q;| = 1} be the unit circle 
in the complex plane. 

Definition 2.4. Let a G T. We write Va C Ti. for the subspace of functions f E Ti. 
satisfying Lf € H, /(0+) = a/(0-) and /'(0+) = a/'(0-). 

Observe that Pq, contains the functions with finite support, hence C 7i is 
dense. 

Lemma 2.5. Let a e T. 

(i) There exists a function f G such that /(O^) ^ and (^Dqf){zq'') = 0, 
{Dj)i-q')^0 fork,leZ+. 

(ii) There exists a function g G such that .g(0+) = and g'(0^) ^ 0. 

(iii) T>a ^ T>/3 for a, /3 G T with a ^ (3. 



Proof. Part (iii) follows immediately from (i) and Definition 2A. For the proof of 
(i), we observe first that the weights of (., .) around zero behave like 

^-^ = Oiq% ^^^=Oiq% k^OO. (2.17) 

Define now the function / G J-{L) hy f{x) = 1 if —1 < x < 0, f{x) — a if 
< a; < z and f{x) = if x > z. Then / G H follows from ( |2.17| ), and Lf e H 
since {Lf){x) = if x ^ {z^q~^z}. Furthermore, /(0+) = a = af{0~) and 
/'(0+) = = af{0-), hence / G X>a. By construction we have /(0+) and 
{DJ)izq^^) = 0, {DJ) (-gO = for fc, ^ G Z+. 

For the proof of (ii) we define the function g G J^{L) by g{x) = x if — 1 < x < 0, 
g{x) = ax if < X < z, and g{x) — ii x > z. Then g E Ti.. Furthermore, 
(7(0+) = = ag{0-) and g'{0^) — a — a.g'(0^). So it remains to show that 
Lg G Ti. Using the explicit expression ( |2.2| ) for the g-difference operator L, we see 
that {Lg){-q'') = 0(1), {Lg){zq'') = 0(1) as k ^ 00. Combined with ( |2T7| ), it 
follows that Lg E H. □ 
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Lemma 2.6. Let a G T, then (L^Va) is a symmetric operator. 

Proof. We have to show that 

{Lf,g)-{f,Lg) = 0, yf,geV^. 

By Lemma 2.2, Lemma p.3| , ( ^.10 ) and the fact that h, Lh G Ti. for h G Va, it 
suffices to show that W{f,g){O^J= W{f,g){0-) for all f,g G P„. Since r(0+) = 
(1 — /qhc = r(0~), we have for all f,g&T>a, 

{l-qf 



W^(/,5)(0+) = 



as desired. 



be 
be 



(/'(0+)5(0+)-/(0+)5'(0+)) 



f(/'(0-)g(0-) - /(O-)g'(O-)) = W{f,g){^-), 



□ 



For a G T, we write (i*,2?*) for the adjoint of the operator {L,T>a). Since 
{L.Va) is a symmetric operator, we have {L,T>a) C 

Recall that L was initially defined as a linear operator on the linear space J- {I) 
of complex-valued functions on /. In particular, L is well defined as a linear map 
L : P* ^{I) by restriction. We claim that 

L*=L|p.. (2.18) 

To prove the claim, we observe that 

{LJ,g)^{J,Lg) 

for functions f,gE ^{I) such that / has finite support (compare with the proof of 
Lemma 2.2). For g G this implies that {f,Lg) = {f,L*g) for functions / with 
finite support. Applying this formula with a non-zero function / hav ing support in 
one point a; G /, we arrive at (^Lg){x) = (^L*g){x). The claim ( 2.1g ) follows, since 
a; G / can be chosen arbitrarily. 

Proposition 2.7. Let a G T, then {L,Va) is self-adjoint, i.e. Va = I?^. 

Proof. Since {L, Vg ) is s ymmetric, it suffices to prove the inclusion P* C V g- Le t 
g G V*^. Then by (|2 18|) , i.g L*.g G H. For any / G Da we have, using ( |2.18|) , 
Lemma 2^, Lemma 2^ and ( 2.10 ), that 



= {Lf,g) - (/,L*5) = M/(/,g)(0-) - W^(/,ff)(0+). 
Now using Lemma 2.5, one derives from the existence of the limits 

W{f,9){0+) ^^^^ yiu,J{D,f){zq^)^)- f{zq%D,g)[zq^)) 



W{f,g){Q-)^ 



{1-1? 
be 



hm {{D,f){-q'^)g{-q'') - f{-q%D,g){^q^)) 



for aU / G I?„ that g{{)+) = ag{{)-) and g'{{)+) = ag'(0"). It follows that g G Vg, 
as desired. □ 

Observe that the operator L G End(jF(/)) preserves the subspace of functions 
/ G ^{T) with support in /_ = [— l,0]g = {~q''}k£Z+, as well as the subspace of 
functions / G ^{I) with support in /+ = [0, oo(z))g — {zq^}i^z- We denote L_ 
(respectively L+) for the restriction of L to functions / G J- {I) with support in 
/_ (respectively /+) and we write Ti. — Ti.^ ® for the orthogonal direct sum 
decomposition where are the functions f ^TC with support in I±. 
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We end this section by indicating the relation of this sphtting of L with the 
possible choices of domains for L. The following results will not be used in the 
remainder of the paper, so we omit detailed proofs. 

We start with the symmetric operator on H+, where C W+ 

is the subspace of functions with finite support. Its minimal closure is given by 
{LjVq ), where 

v+ = {fGn+\ L+f e n+, /(o+) = /'(o+) = 0}, 

and the deficiency indices of {LjVq) are (1, 1). 

The spectral properties of the self-adjoint extensions of (L+j'Dq) are highly 
sensitive with respect to the choice of self-adjoint extension. Furthermore, the 
spectral analysis of Z/+ with respect to a fixed choice of self-adjoint extension seems 
to result in a rather implicit description of the spectrum and of the resolution of 
the identity. 

In order to obtain better spectral properties, we "blow up" the Hilbert space 

7i+ to Ti = 7i_ ® and wc regard L now as an unbounded operator on Ti. with 
domain Vfin given by the functions in H with finite support. Its minimal closure 
is given by {L,'Do), where 

Vo = {f€'H\Lf&H, /(0+) = /(0-) = = /'(0-) = /'(0+)}. 

The closed, symmetric operator {L, Vq) has deficiency indices (2, 2). By the increase 
of deficiency indices we have a larger choice of self-adjoint extensions for (L, Vq) 
than for {L^,Vq). In particular, the self-adjoint extensions {L,Va) {a e T) are 
self-adjoint extensions for which {L^,Va H 7i+) are not self-adjoint. 

It turns out that the spectral analysis of {L,Va) is essentially independent of 
a G T, so we restrict attention in this paper to the unbounded self-adjoint operator 
L on Ti with domain of definition T? = T>i C Ti.. In particular, any function 
f GV is continuously dijferentiable at the origin, i.e. / satisfies /(O"*") = /(0~) and 
/'(0+) = /'(0-). 

In the next sections wc show that (L, V) is a self-adjoint extension of (L, Vq) for 
which the spectral analysis can be derived in a very explicit manner. The reason is 
that for a given generic eigenvalue, one has two linear independent eigenfunctions of 
L which arc explicitly given in terms of basic hypcrgcomctric series and which are 
continuously differentiable at the origin. The corresponding Wronskian, as well as 
their Wronskian with the asymptotically free solutions of the corresponding eigen- 
value equation, can be computed explicitly. This allows us to derive the spectral 
properties in a very explicit manner. 

Let us finally make some remarks on the "blowing up" procedure of the Hilbert 
space Ti.^ as described in the previous paragraphs. It is a well known principle in 
harmonic analysis that the spectral properties of the unbounded self-adjoint oper- 
ator under consideration is essentially determined by the behaviour of the eigen- 
functions at infinity (along the support of the measure). The fact that 7i_ is a 
L^-space of functions supported on a compact space implies that the situation at 
infinity does not alter in the enlarged setting. Hence, the spectral properties of the 
extension of to the unbounded operator L on H. should be essentially deter- 
mined by spectral properties of L+. We will justify this principle in this paper by 
deriving most of the spectral properties of L from the spectral properties of Z/+. 
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The blowing up of the Hilbert space 1-1+ is canonical in the sense that the extra 
piece Tl- added to the Hilbert space 7i+ is essentially determined by the following 
three properties: 
(i) The support of the measure of is a hounded q-interval, 
(ii) The operator {L,'D~^^^) on is symmetric, where T^^i^ consists of functions 

in with finite support, 
(iii) The weight function is continuous at the origin. 

Indeed, the condition that the support of the measure oiTL" is a bounded g-interval 
implies that the support is of the form [t/, 0]^ = {yq^}k£Z+ with B{y) — 0. This 
leads to the possibilities y — —1 or y — —bc/q. Choosing y = — 1, we derive from 
condition (ii) that the weight function corresponding to Ti." is given by l/p(-) up 
to a positive constant, while condition (iii) implies that the constant is one. Since 
p{x; a, b, c) = p(bcx/q;a,q/b,q/c) and a similar property holds for L, see Remark 



3.4, we may take y = —1 without loss of generality. 

3. ElGENFUNCTIONS OF L 



In sections we consider L with domain of definition D — Di. In particular, 
any function / in the domain of definition V is continuously differentiable at the 
origin. We set /(O) (respectively /'(O)) for the common limits /(O^) (respectively 
/'(0±)). 

Furthermore, we need to restrict sometimes the choice of parameters (a, b,c) ^ V 
to a dense subdomain Vf"^, which is defined by 

y/™ = {(a, b,c)eV\ a^, 6^, c^, ab, ac, be, a/b, a/e, a^b^e^z^ ^ {q^}kei}- (3.1) 

In this section we consider eigenfunctions of the linear operators L and L±, and 
study their behaviour at the origin. We write (—1, 0]q — I- \ {—1} = [—q, 0],, and 
define for /i e C the linear spaces 

= { / : /± ^ C I L±/ = /i/ on /± }, 

= {/:/- ^ C I L_/ = /i/ on (-1, 0], }, (3.2) 
= {/ e I Lf = fif on (—1, oo{z))q, f cont. differentiable in 0}. 

Observe that V.'T C V,r . 
Lemma 3.1. Let /i G C. 

(i) dim{V+) = 2, dim{V-) = 1 and dim{y-) ^ 2. 

(ii) ///i,/2 S [respectively /i,/2 S then W{fi,f2) is constant on 
{respectively constant on /_). 

(iii) ///i,/2 S Vfj^, then Vt^(/i,/2) G ^{I) constant on I. 

Proo f, (i ) We start with computing dim(V^~). The coefficient A{x) in the expres- 
sion ( |2.2| ) of L is non-zero for a; G /_ since (a, 6, c) € V . It follows that solutions 
of i_/ = /i/ on /_ are in one to one correspondence with solutions {ak)ke'L+ of a 
recurrence relation of the form 

^Ofc = akOk^i + Puak + IkCLk+i, 7fc 0, ke'L+ 

with a_i = by definition. The correspondence is obtained by associating the 
sequence Ok — f{—q'') {k £ Z-|_) to / S V~ . The coefficients of the recurrence 
relation are then given by ak — —B{~q^), (5k — '^A{~q^) — B{—q^) and 7^ = 
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A{—q^). Any solution of such a recurrence relation is uniquely determined by 
ao — /( — I) £ C, hence dim(y~)— 1. 

Then dim{V^)= 2 follows from the fact that functions / £ V~ are in one to one 
correspondence with solutions {ak)kez+ of the recurrence relation 

^flfe akttk-i + hak + TfeOfc+i, Ik ^0, fc £ N. 

Finally, in order to show that dim(V^j^)= 2, observe that solutions of i+/ — /i/ 
on are in one to one correspondence with solutions {ak)kei^ of a double infinite 
recurrence relation of the form 

^ittk = akUk-i + Pkttk + 7fca/c+i, a/c 7^ 0, 7fc 7^ 0, A: 6 Z, 



since and (2.3) in the expression (^^) of L are non-zero for a; G /+. This 
is a two-dimensional space. 

(ii) Let /i , /2 G -^l^) • Using Lemma |2.l| , the product rule for the g-derivative 

{D,{fg)){x) - {Dj){x)g{x) + f{qx){Dgg){x), (3.3) 
and the second equality of ( |2.11| ), we have for all x G (—1, oo(z))g, 
q-'p{x){DgW{hj2)){q-'x) 

= {Lh){x)h{x) +p{x)r{q-^x){D,h){q-^x){D,h){q-'x) 

- {Lh){x)h{x) ^p{x)r{q-^x){D,h){q-^x){D,h){q-^x) 

- {Lh){x)h{x)-{Lh){x)h{x). 



If /i,/2 G V;-, then it follows from (|j) that {DqW{h, f2)){x) = for aU 
X G (— l,0]g since p{x) 7^ (a; G /-), hence M^(/i,/2) is constant on A 
similar argument shows that VF(/i, /2) is constant on /+ if /i, /2 G V^. 
(iii) Let /i,/2 G V^. The continuously differentiability of the /j's at the ori- 
gin yields that W^(/i, /2)(0^) = W^(/i, /2)(0~). Indeed, both sides are equal to 
(jr(0)(/{(0)/2(0) - /i(0)/^(0)) (compare with the proof of Lemma U). The resuh 
follows now from (ii). □ 

Observe that /|/_ G and /|/_^ G V+ for f eVfj,. 

Proposition 3.2. We have dim{Vfi) < 2. Furthermore, the following three state- 
ments are equivalent: 

(i) The linear map Vp. ^ is a bijection. 

(ii) The linear map / 1^ /|/^ '■ ^ is a bijection. 

(iii) dimiV^) = 2. 

Proof. Suppose that dim(yp) > 2, then we claim that the map / ^ /|/^ : V^^ — > 
is injective. Suppose that the map is not injective. Then there exist two 
linearly independent functions /i,/2 G such that /i | /_,. , /2 1 G are linearly 
dependent. The Wronskian Vl^(/i,/2) is not identically zero as function on / by 
the linear independence of /i and /2. Since Vl^(/i,/2) is constant on / by Lemma 
3.1(iii), it follows that VK(/i,/2) is not identically zero on /+, which contradicts 
the linear dependence of /i | and /2 1 . 

So if dim(y^) > 2, then it follows from Lemma pT| (i) and from the injectivity 
of the map / 1-^ /|/^ : V+ that dim(y^) = 2. This proves that dim(V^) < 2 

for all /i G C, and it proves the implication (iii) (ii). The implication (iii) => (i) 
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is proved in a similar manner, while the implications (i) (iii) and (ii) => (iii) are 



immediate consequences of Lemma 3.1 (i). □ 



Corollary 3.3. // dim(V^) = 2, then any g ^ {respectively g G V^) extends 
uniquely to a function g E V^. 

In the remainder of this section we introduce two explicit functions which live in 
V^. We define for 7 £ C*, 

/i(7) = -l-a2 + a(7 + 7"'), (3.5) 

and we write Xn — (bcx^ /q)"^ (^—q/bcx, —l/x] qj^g-y{xq^"') where g^ is a function 
satisfying {Lgj){xq^") — fj,{j)gj{xq^"). Then the Xn satisfy the three term re- 
currence relation (2.1)] with parameters {A, B,C, D, z) in H, (2.1)] given by 
A = —q/hcx, B — —l/x, C — —q/acx, D — —qjahx and 

abcx^ , 1, q , , abcx^ i, 

^ - + ^ ) - ABCDXZ + ^ 

In ^ it is was shown that the three term recurrence relation (2.1)] is the one sat- 
isfied by the associated continuous dual g-Hahn polynomials. Furthermore, several 
explicit solutions of the recurrence relation (2.1)] were derived explicitly in 
section 2]. In particular, the explicit solution g, (2.13)] of the recurrence relation 
§ (2.1)] implies that 

(j^-yix) — 4i^{x;a,b,c) =3^2 ^ ' '^ab 'ac^ ^ ^ ^'Zi^^cx^ , \bcx\ < 1 (3.6) 

satisfies (_L0-y)(a;) = fi{'y)(j)j{x) for x G {—l,Qo{z))q with \x\ < q/bc. Observe that 
(p-y is well defined since (a, 6, c) £ V. 



The explicit solution ]|9|, (2.13)] of the recurrerence relation |9|, (2.1)] implies that 

, / 1 \ if a'y,a/^,—qlbcx \ 1 1 -, ,^ „\ 

ipj[x;a,b,c) = 302 ( ^^'^^ ^^j^ \q,-qx I , \qx\ < 1 (3.7) 

satisfies (L'f/'-y) (x) = iJ.{'y)ip~f{x) for x G (— l,oo(z))q with \x\ < 1 when (a, 6, c) G 
ygen ^ 

Since be < 1, we have that and -0^ are well defined on /_ and that they are 
solutions of Lg = iJ.{"i)g on (—1,0],. Since (j)-^ and 1/^7 are solutions of (^Lg){x) = 
fi{'j)g{x) for X G 1+ with x < 1, it follows that (f)^ and ip^ uniquely extend to 
functions on / satisfying the eigenvalue equation Lg — fJ-{"f)g on {—1, oo{z))q, cf. 
the proof of Lemma 3.1(i). For given x G /, (t>-y{x) and ip-yi^x) depend analytically 



on 7 G C* and are invariant under 7 <-> 7^^. 

The extension of (f>^ to a function on / can also be obtained by the transformation 
formula ]0, (3.2.10)], which yields 

, , . {aj,bc,-abcx/r,q)^ f b/-f,c/j,-bcx \ ,„ 

'^7(2^) = / . N 302 ' , , , ; 9, 07 . (3.8) 



(ab, ac, -bcx; q) ^ \ bc,-abcx/^ 

This gives a single valued analytic continuation for (j)-y{x) for 7 G C* with ]7] < 

to X G C \ (-00, -l/bc\. Observe that (-00, -1/bc] fl / = since (a, 6,c) G V. If 

a < 1, then the analytic continuation for I7] > can be obtained by replacing 
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7 by in the right hand side of (3.8). If a > 1, then 6 < 1, and we can use 
(3.2.7)] to obtain 

which yields then the exphcit expression for the analytic continuation of cf)^ (7 G C*) 
in a similar manner as for o < 1. A similar remark holds for the solution ij]^. 

Remark 3.4. If /(•) = f{-;a,b,c) satisfies the eigenvalue equation Lf — ^.{'^)f, 
then g{x) = f{hcx/q\ a, q/b, q/c) satisfies the same eigenvalue equation. It is easy to 
check using the explicit expressions (3.6) and (^^) for (p-^ and V'7 that the solutions 
and il)^ are interchanged by this symmetry, i.e. 

■07(2;; a, 6, c) = (f)-y(bcx/q; a, q/b, q/c). (3.10) 
From ( |3.6| ) we see that 0-y(— 1) = 1 and by direct computation we have 

{Dq(t)^){x;a,b,c) = _ q^(i^^^^ab){l - ^' x e I. (3.11) 

It follows that 

so {L(j)^){-1) = /i(7)0^(-l) by (^^. We conclude that <j)y\i_ £ V~^^^ for aU 
7 G C*. For ipj we have the formula 

{D,i.,)ix;aAc) = (1 _ ^'.'^^(i _ ^ c). (3.13) 

We stress already the fact that in general we have [Lip^){—1) ^ fi{'y)ipj{—l), so 
that G V^{^) \ V^(7), see Corollary |5.2|(iii). 

Lemma 3.5. Let 7 G C*. The function (j)^ G J' {I) is continuously differentiable at 
the origin, i.e. (p^ G In fact, we have 



a-/, a/ J 



(3.14) 



'^^^'^^ ^ {1 - q)il - ab)il - acf ^' [ qab,qac ' 
For (a, 6, c) G Vlf^" we /laue ip^ G V^i(7) 

&c 

■07(0; a, b, c) = 0^(0; a, 9/c), '(/'7(0; a, c) = —'^7(0; 9/^' 9/^). 

where we have extended the definition of 4>y{0) and (t>j{0) to generic parameters 
(a, 6, c) G C^"^ by analytic continuation of the right hand sides of ( |3.14 ). 

Proof. The proof for is a direct consequence of (B.ll) and the explicit expression 
(3.£) for (f>j. The proof for ip^ follows then from ( |3.7| ) and ( [3.13 ). □ 

In section H we will evaluate the Wronskian W(jp^, ip-y) explicitly, see Proposition 
5.1. In particular, this will give explicit criteria on the spectral parameter 7 for 
which we have dim(V^(^)) = 2, i.e. for which Corollary |3.3| is applicable. But first 
we need to study yet another solution of the eigenvalue equation Lg = fJ.{'y)g, the 
so called asymptotic solution. 
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4. The asymptotic solution 

In this section we determine the asymptotic solution of the eigenvalue equa- 
tion L^g = fJ-{'y)g on I^. We furthermore determine the c- function expansion of 
(/)-y on /+, i.e. we write (p-yli^ explicitly as linear combination of the asymptotic 
solutions <i>^ and <I>^-i. 

We define singular sets by 

5+„^ = {±q-^%^n, S,.„g = {ig^^fea, (4.1) 

and we write S^^g, Sreg for the complements of these singular sets in C*. The c- 
function expansion of (f)^ on which we derive in this section, holds for 7 G Sreg- 
We consider for y > the minimal solution (2.26)] of the three term recurrence 
relation (2.1)] with corresponding parameters {A, B, C, D, A_) given by 

A = -q/bcy, B = -q/aby, C = -q/acy, D = A_ ^ 7- 

This leads for 7 G S^^g with I7I < to the following explicit solution ^^{x) = 
^{x; a, 6, c) of Lf = ii{-f)f on [0, oo{y))q, 

^y^^^ _ (-97/ ax, -g^7/ abcx, 07; q) ^ 
{-q/abx,-q/acx,qj^;q)^ 

/ \-k , f q^ I a,—ql abx,—ql acx \ i. 

■(«^) ^^^[ qj/aLq^,/abcx ' «^ j ' ^ = ' 

For given x € [0, oo{y))q, we have a single valued analytic extension of ^'^{x) to 
7 G Indeed, for a; G [O,oo{y))q with x > fec/q we can apply (3.2.7)] to 

arrive at 

^y^^^ _ {-q/bcx,-qj/ax;q)^ 
^ {-q/abx,-q/acx;q)^ 

which gives the analytic extension in this case. For x G [0, oo(y))q with y < bc/q we 
can use the eigenvalue equation — ^{-f)^^ on [0, oo{y))q to derive the analytic 
extension for $^(a;) from the analytic extension of <I>^(m) with u > bc/q. It follows 



from (O) that the asymptotics to infinity of $^ is given by 

$^^(2/9-") = (a7)"(l + 0(g")), m^oo. (4.4) 

Definition 4.1. Let 7 G 5,+ ^. We call ^j(-;a,b,c) ~ $^(-;a, 6, c) i/ie asymptotic 
solution of L+f ~ ^J'{'y)f on — [0, oo(z))g. 



Definition 4.1 is justified by the following lemma. 

Lemma 4.2. Let ^ G C* and K = {zq^^}kei.+ C /+. S'ei 

Af^ = {/i : /_(_ ^ C I L^h = fih on L^ and Wh^W"^ < 00}, 

where hx S ^{I) is defined to be equal to zero on L \ K and to be equal to h on 
K . Then = span{$-y} for all ^ G C\^, where 7 G C* is the unique non-zero 
complex number such that = /i(7) and I7I < 1. 
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Proof. First of all, observe that if 7 G C* and I7I < 1, then 7 G S^^g, hence $^ is 
well defined. 

Let fi G C \ R and let 7 G C*, I7I < 1 be such that /i = ^(7). Using the 



asymptotics (2.13) for the weights of the inner product (., .), it follows from (4.4) 
that G M^(^^) because I7I < 1. 

It remains to show that dim(Af^t) = 1. For the proof we use some well known 
results from the theory of the classical moment problem, see for instance or |^ . 
If / : /+ ^ C satisfies i+/ = /i/ on /+, then by setting 



a-k = .f{zq 



we see that (ak 



ith 



feez 



satisfies the recurrence relation 

afc-iflfe-i + PkO-k + oikak+i = 



A: G Z, 



(4.5) 




and (3k — —A[zq 



B{zq- 



It follows from Corollary 4.5] and from the 
fact that the sequence (ak),^.^ is bounded that the Hamburger moment problem 
corresponding to the recurrence relation (L5) for k G Z_|_ is determined. By [ pT| 



Theorem 3] this implies that dim(A/^) — 1, as desired. 



It follows from Lemma 4.2 that 



4'?,'=^/«(6ca;/g; a, q/b, q/c) = $^(a;; a, 6, c), 



x G A 



□ 



(4.6) 



for parameters (a,6, c) G V such that {a,q/b^q/ c) G V. Indeed, both sides of (4.6), 
considered as function of a; G /+, are solutions of L+/ = u{'j)f on /+ by Remark 
3.4 , and they have the same asympto tics to infinity by (L4). Formula (4^) is also 
obvious from the explicit expression ( |4.2| ) for <I>^. 

Lemma 4.3. For 7 G Sreg o,nd x E I+, we have 

iy($^,$^-i)(a.)-aX(7-7"') 7^0, 



where K is the positive constant defined by (2.14). In particular, {$^,$..^-1} is a 
basis of V"^^^ when 7 G Sreg ■ 

Proof. The explicit expression for the Wronskian follows by computing the limit 
lim„i^oo W{^j, ^^-i){zq~"^) using the first expression of ( ^.11 ) and the formulas 
( pl5| ) and (Q. Since the Wronskian is non-zero, it follows that and 'I'^-i are 
linear independent, hence they form a basis of V^^^-j by Lemma 3A(i). □ 



It follows from Lemma 4.3 that and i^-yli, can be written uniquely as linear 



combination of <I>^ and 



for 7 G Sreg- The corresponding coefficients can be 
expressed in terms of the c-function 0(7) = 0(7; a, 5, c; z), which is defined by 



c(7) 



1 



(a/7, 6/7, c/7; q)^6{-qlabcz-i) 



(ab, ac; q) 6{-bcz) 



(1/7^; 9) 



(4.7) 
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Proposition 4.4. Let 7 e Sreg- Then we have 

(j)^{x) = c(7)$T,(x) + c(7"^)$-y-i(a;), x e /+. 
The same formula holds for ^j{x) when {a,b,c) G V^"^"" , with 0(7) replaced by 
c(7; a, b, c; z) = c{j; a, q/b, q/c; bcz/ q) 

_ 1 (a/7, q/b-/, q/cj; q) ^9{-l/azj) (4.8) 

" {qa/b,qa/c;q)j{-l/z) 

Proof. We first prove the connection coefficient formula for 0^. Observe tliat 0(7^^) 
is well defined for 7 G Sreg since (a, 5, c) £ F. We fix a; = zq'' with fc £ Z such that 
q/acx < 1. Furthermore, we assume that a < 1 and we fix 7 S C \ (— oo,0] such 
that 7 e Sreg and a < I7I < 1/a. 

By the assumptions on x and 7, we may apply the three term recurrence relation 
!?[ (3.3.3)] with a — > —bcx, b — > 6/7, c — > c/7, d — > —abcx/^y and e — > 6c. We arrive 
at 

/ ~bcx,b/j,c/-f 
'^'[ bc,-abcxh ''^'''^ 

{c-i,b-i,-q/bx-i,-q-t/abcx;q) f cl-i,ah,q/b-f . 



{bc,-qlabx,-qlbcx,i^\q)^ \ -(ll^^l.flh 

_ {-qi/abcx, -qj/ax, b/^f, c/7, a/7; q)^e{-abcxj/q) 

iy—abcx/q^, be, —q/acx, —q/abx, —q/bcx; q)^6{'-f'^) 

( q-f/ a, -q/acx, -q/abx 

-q^l/abcx,-q^/ax ''^'"^ 

The 3(/)2 on the left hand side is the 3^2 in the expression ( |3.8|) for (f>^ and the 
second 3^2 on the right hand side is the 3(^)2 in the expression ( |4.2| ) for $-y. Again 
by the assumptions on x and 7, we may rewrite the first 3(^12 on the right hand side 
using iQ, (3.2.10)] with a — > c/7, b —* a/"1, c — + 9/67, d — > (7/7^ and e — > —q/bx^. 
This gives 



39^2 1 „ /„2 ■,q, -q/acx 



c/l,a/-f,q/b-f _^ 
-q/bx-i,q/-{^ 

_ (a/7, -q^/abcx-i, -q/axr, q) ^ ( q/a-f, -q/abx, -q/acx ^ , \ 

{q/l'^,-q/bx-i,-q/acx;q)^ ^'^^ V -Q^ / abcx-i,-q/ ax-f ' ^' "/^J 



The 3(/)2 on the right hand side is the 3^2 in the expression (4.2) for $-^-1. So 
substituting this formula in the three term recurrence relation, and simplifying the 
formulas using in particular the functional relation 

for the Jacobi theta function, we arrive at the desired result for restricted choices 
of X, a and 7. The extension to all a; G /+ is made using the fact that the left 
hand side and the right hand side of the c-function expansion are solutions of the 
eigenvalue equation L+/ = ^(7)/ on /_|_. Finally, the restrictions on a and 7 can 
be removed by analytic continuation. 
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The proof of the connection coefficient formula for ■0^ foUows by analytic con- 
tinuation from the connection coefficient formula for (jy-^ using ( P^ ) and Q. □ 



For (a,6, c) £ V"/™ and 7 e S'+g with I7I < a ^ we define ^^{x) for x e 
(oo(-l),0], by 

{-q'^-f/abcx,-q'y/ax,a'y;q)^ 



{-q/abx, -q/acx, qj'^; q) ^ 
I I ( qi/a.—q/abx.—qlacx \ u 

Observe that <i>~ is obtained by taking y = — 1 in the definition of the eigenfunction 
see (gj). 



Lemma 4.5. Let {a,b,c) e V^"^ and 7 e S'+g with I7I < a ^. T/ien 

{qlab,q/ac,qY]q)^ 

Proof. Using the minimal solution (2.26)] of the three term recurrence relation 
|9[ (2.1)], it follows that is a solution of Lf = fJ-{j)f on (oo(— l),0]g, where 
L is the second order g-difference operator defined by ( ^.2| ). In particular, we 
have A{-l){^-{-q) - $:7(-l)) = since B(-l) = 0. It follows that 



'1 \1- ^ V(7)- 



We have dim(V^^^^) = 1 by Lemma 3.1 (i) and 7^ (/)^|/_ £ ^(7)- follows that 



$^ |/_ — C^(j)^\i_ for a unique constant C^. The explicit expression for can 
be found using (/)^(— 1) = 1 and by evaluating $~(— 1) using the g-Gauss sum 
(1.5.1)]. □ 



Observe that the Wronskian VF(0^, $^ ) is identically zero on /_ by Lemma L5 
On the othe r hand, the Wronskian W{(t)^T^^) on 1+ is non-zero for generic 7 by 



Lemma ^.3| and Proposition [4.4 Suppose now that for a given (generic) 7 G C* 
with I7I < a^^ we have an extension of € ^^^) to a function <I>.^ e ^(7)- Then 



the Wronskian W{<j}^, $^) is constant on / by Lemma 3.1(iii) and Lemma 3.5, so 
$^|/_ is not a multiple of <I>~ on 

This shows that the extension of $^ e K^7) ^ function $^ e ^Ai(7) *^^u not 
be obtained by taking the explicit expression for <I>^ and extending its definition 
in an obvious manner to the whole g-interval /. We derive in the next section the 
extension of $^ S ^7) ^ function e V^(7) for (a, 6, c) G y/'^" by writing 
explicitly as a linear combination of the two eigenfunctions (j)^ € ^m(7) ^-'^'^ 

V'7 G "^'/^(7)- 

5. The extension of the asymptotic solution 

In this section we first evaluate the Wronskian Vl^('(/i^,0^) explicitly in product 
form. From this evaluation we derive explicit conditions on the spectral parameter 
7 for which we have dim(V^(^)) = 2. For these values of the spectral parameter. 



there exists a unique extension of $^ which lives in V^(7) by Corollary 3.3. We will 
give this extension explicitly as a linear combination of (j)-^ and . 
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Observe that if we compute the Wronskian by taking the hmit to 0, 

[l-qf 



be 



■(v;;(o)0^(o)-7^^(o)0;(o)), 



and by applying Lemma |3.5| , we get an explicit expression as a linear combination 
of products of 202 's, while evaluation of W {^l^^ ^ 4>~f){x) at a; = —1 using (3.12) and 
<j)~f{—l) ~ 1 gives an expression of the Wronskian as a linear combination of two 
3</)2's. From both these expressions, the (non) vanishing properties of the Wronskian 
Vr(^-y,0-y) are hard to derive. In the next proposition we evaluate the Wronskian 
by substitution of the c- function expansions for (p-^ and ijj-y. 

Reca ll th at the Wronskian W{ip^,(t)^') is constant on / by Lemma 3J(iii) and 
Lemma 3.5. 

Proposition 5.1. Let 7 e C* and {a,b,e) G V^/"^", then 

ia-f,a/j;q)^e{bc) 



W{ij^,cj)^) = {l-q)- 



{ab,ae, qa/b,qa/e; q)^ 
Proof. We first assume that 7 G Sreg ■ Using Proposition [4.4| we then have 
W{^^,^^) = cij)eij-')W{^^,%-i){z) + d{j-')eij)W{'P^-i,^^)iz), 



where c(-) and c(-) are given by ( |4.7| ) and ( |4.8| ), respectively. By Lemma | 
right hand side of (^.2D is equal to 



W{^P^,cj)^) = ^aK (c(7)5(7"') - c{j-^)c{l)) (7 - 7^'), 



(5.1) 

(5.2) 
I the 

(5.3) 



where K is the positive constant defined by (2.14). It follows by direct computation 
that 

1 



c(7)S(7 ) - c(7 )5(7) 
(07,0/7;'?). 



(7^7 ^;9) 



(a6, ac, qa/b, qa/e; q^ ^9{—q/bez, —l/z) 
{0{q'y/b, c/7, -q/abez-f, ~-f/az) 



—0{q/b^, c^l, —q^/abez, —1/027)} • 
Now we can apply the 0-product identity 

9{x\, x/X, fii^, ^/v) — 9{xv, x/v, A/1, ^/\) — ^9{xiJ., x/n, Xv, X/y), 

X 

see [0, Exercise 2.16], with parameter values 



(5.4) 



6c' 



A = 7, 



be' 



to obtain 

c(7)c(7~i) - c(7~i)c(7) = 



az V be' 

-1 



7 V 6c' 



(a6, ac, qa/b, qa/e; q) ^9{~-q/bcz, -l/z) 
1 (07, a/7; 9) „ 



(5.5) 



■az7 (72,7"^;?), 



-q/abz, —aez, q/be, 7 ). 



The proposition now follows for 7 G Sreg by substitution of ( p.5| ) in (5.3) and using 
9{x) — 9{q/x). By continuity in 7, it follows that (5.1) holds for all 7 G C*. □ 
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In the following corollary we give a characterization of the eigenfunction (j)^ 
of L for generic values of the spectral parameter 7. In section ^ we show that 
the eigenfunction 0^ plays the role of the spherical function for the big q-Jacobi 
function transform. 



Corollary 5.2. Let 7 e C* and (a, 6, c) e V^/™. 
(i) {4>~f,ijj^} is a linear basis o/y^(^) if 



(5.6) 



(ii) Assume that (5.6) is satisfied. Let f G ^{I) be a function satisfying 

- LJ ^ fi{j)f on I; 

— f is continuously differentiable at the origin; 

-/(-1) = 1- 
Then f = 



(iii) // (5_^) is satisfied, then (L^-y)(— 1) 7^ 



Proof. We fix 7 e C* satisfying jplq). Th en the Wronskian W{ip'yi4''y) is non-zero 
since (a, b, c) G V^^, see Propo sitio n 5.1 . Hence {0^, tp-^} is a linear basis of ^^(7) 
by Proposition |3.2| and Lemma |3.5| . 

For the proof of (ii), we observe first that (j>^ satisfies the three properties as 
stated in (ii), see section |[ If / is another function satisfying the same three 
properties, then f\j_ — (j)-y\i_ since /(—I) — (t)j{—l) = 1 and dim(V^^^j) = 1 by 
Lemma 3.1. Hence f = cj)^ on I by Corollary 3.3. 

Finally, for the proof of (iii), observe that tpj\i_ and 4>^\i^ are linearly inde- 
pendent by part (i) of the corollary and by Proposition Since (l).y\i_ G ^^^^7) 
by part (ii) of the corollary and dim(y^^^j) = 1 by Lemma ^]^(i), it follows that 
e V, . \ V,y So {L^p^){-1) ^ m(7)^7(-1)> as desired. □ 



^(7) ^ ^(7) 

Set 7„ = aq^ (n G Z+). We have the following description of the excluded set 



in Corollary 5.2 



Proposition 5.3. Let Spoi C C* be the set of spectral parameters 7 G C* for which 
the eigenvalue equation {Lf^{x) = iJ,{'y)f{x) on I has a solution / ^ which is 
polynomial in x. Then Spoi = {'jt^}ne2+ ■ The non-zero polynomial eigenfunction 
corresponding to the eigenvalue /i(7,^^) is the big q-Jacobi polynomial of degree n 
and is explicitly given by 



^7. (x) 



{qa/b, qa/c; q)^ 
{ab,ac; q)^ 

[gale; q) 
(ac; q)^ 



aq 



(n+l)/2 



—abx, q^a^ 
ab, qa/c 



.q,q 



(5.7) 



Proof. This is well known, see for instance and section 7.3 ] . The connection 



between (p^^^ and -^^^ with the 302 in the last equality of (5.7) follows from 
(3.2.5)]. " " □ 



Remark 5.4. Proposition 5.1 and Proposition 5.3 are essential ingredients for a 
functional-analytic derivation of the orthogonality relations and the quadratic norm 
evaluations for the big g- Jacobi polynomials, see section nG. 
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We write S'jf^j = {7„ }«6Z+, where 7„ = aq". Observe that S'+^j C Spoi C Sreg C 
S+ when (a,5,c) S F/'^". 



Proposition 5.5. Let 7 e S'+g \ 5'+, and (a, 6, c) e y/™. T/ien 
= i^(7)(/)^(a;) + ii:(7)V'^(x), a; e /+, 
wt/i ^^(7) = K{'j; a, 6, c; z) given by 

^^^^ _ (ab, ac, q-i/b, q-y / c] q) ^ Oj-bcz, -az/j) 
(97^0/7;'?) 00 9{bc,-abz,~acz)' 
and K{'j) — K{'j; a, b, c; z) — K{j; a, q/b, q/c; bcz/q). 



(5.8) 



Proof. We first prove ( |5.8| ) for 7 e S',.eg \ S'po;- By Corollary 5.2(i) and Proposition 
3.2, there exist unique 1^(7), if (7) G C such that (5.8) holds for all x G /+. These 
coefficients can be expressed in terms of Wronskians by 



Kil) 



Wi4'^,'f^)iz) 



W{<^^,q^^)iz) 



(5.9) 



(5.10) 



By Lemma 4.3 and Proposition [4.4| we have 

W{i;^,'S>^){z) = aifS(7-i)(7-i - 7), 

W{%,(f>.,){z) = aKc{j-'){-f - J-'), 

where K is the positive constant defined by ( |2.14 ). Formula (5.8) now follows by 
substituting (5.10), the explicit formula (5.1) for the Wronskian W{il'-y, (fij), and 
the explicit expressions ( |4.7| ) and ( ^^ ) for 0(7) and £(7) in (5.9), and using the 
theta function identities 9{x) = 0{q/x) an d ([4.9[ ). 

□ 



It follows now by continuity in 7 that ( p.8\) is valid for 7 G 5*+ „ \ S'Z, 



Remark 5.6. F suppose that {a,b,c) G V"/*^" . The proof of 

Proposition 5J implies the matrix equation 

'c(7) c{j-')\ ^ ( Kij) K{j) ' 
,c(7) 5(7-1)^ \K{j-') X(7-i), 



(5.11) 



Indeed, ( ^.11 ) follows easily from the explicit formula (5.5) for the determinant of 
t he m atrix on the left hand side of ( 5.11 ). Let x G /+, then the matrix equation 
( 5.11 ) implies that the two connection coefficient formulas 

(j)j{x) =c(7)$^(a;)+c(7~^)$^-i(a;), ip^{x) = 5(7)$^(x) + 5(7-^)$^-! (a;) 

are equivalent to the two connection coefficient formulas $-y±i (x) = K{'-f^^)(f>j{x) + 

K(7±i)V7(^)- 

Corollary 5.7. Let 7 G 5,+ ^ \ Spoi and (a, 6, c) G V"/^". The unique extension of 
$7 G ^^^) to o- function G ^^^(7) is given by 

%{x)^K{j)(f>^{x)+K{j)i;^{x), xel, (5.12) 

with if (7), if (7) as defined in Proposition [^T 



Proof. Immediate from Corollary 3.3, Corollary |5.2|(i) and Proposition 5.5 



□ 
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In section || we have seen that ^^(a;) depends analytically on 7 G „ for 
all X e /+. We have the following analogous result for the extension ( 5.12 ) of 

$^ e V+, 

Lemma 5.8. Let 7 G Spoi and {a,b,c) G y/'^". There exists a unique extension of 
$7 G to function G 1/^(7) suc/i t/iai 7 1— > $7(2;) is analytic at j = j for 

all X £ I . 

= In^ G ^poi' then the extension of is given by (5.12). If J ~ Jr 
then the extension of $-y„ is given by 



^7.(2;) 



ResK{-f) 

7=7n 



d-f 



{x)\j=j^ + Mn(j}j„{x) + 



ResK{-f) 

7=7>i 



'^(.)l7= 



7=7n 

(5.13) 



for X G /, mi/i A/„ given by the existing limit M„ = lim^^-,.^ Mn{"/), where 

{qa/b,qa/c; q) 



M„(7) = 



{ab, 



ac; q 



-] if (7) 



if (7). 



(5.14) 



Proof. The proof for 7 G 5"^; is trivial since if (7), if (7), 4"ri^) ''Pyi^) 
regular at 7 = 7. 

For 7„ G S'j^j (n G Z+), observe that if (7) and if (7) have simple poles at 7 = 7„ 
and that 4'-y{x) and '07(2;) S'l'e regular at 7 = 7„. It follows from (5.12) and the 
first equality of (|5.7|) that the singularity of ^"7(3:) at 7 = 7„ is removable for x G i 
if the (at most simple) singularity of Af„(7) at 7 = 7„ is removable. This can be 
checked by direct computions using the theta function identities 9{x) — 9{q/x) and 
(0). 

It follows that lim. 



of <I>7„ hes in V^(7^ 



7_7„ Mn{'^) exists and that ( 5.13 ) holds. The extension ( ^.13 ) 
because (/j-y^ and the derivatives of (j)-y and V'7 a-t 7 = In are 



continuously differentiable at the origin, cf. Lemma 3.5 



(^7 



□ 



For future reference, we collect here the main results concerning the asymptotic 
solution of the eigenvalue equation Lf = /i(7)/. 

Theorem 5.9. Let x e I and (a, 6, c) G y/™. 

(i) $7 G V^(7) for allje S+g. 

(ii) $7(2:) is analytic 7 G S^g- 

(iii) $7 G P /or 7 G C* with \-f \ < 1. 

(iv) (j)j{x) = C(7)$7(x) +C(7"l)$^-l(x) for all-/ G Sreg- 

(v) Let 7 G iS^g. The Wronskian VF(7) = VF(<I'7,07) is constant on I, and 
14^(7) — aif 0(7^"'^) (7 — 1^^)> where if is the positive constant defined by (2.14). 



Proof, (i) and (ii) follow from Corollary 5.7 and Lemma 5.E. 

(iii) Let 7 G C* with I7I < 1, then 7 G S^g, hence $7 is well defined. Then $7 G 



follows from part (i) of the theorem and from Lemma 4.2 



(iv) We first prove the connection coefficient formula for 7 G Sreg \ Spoi. Then 

Since 



4.4 



the connection coefficient formula is valid for all x G J+, see Proposition 
(p-y, $7±i G Vu(7), it fol lows by the u niqu eness property of extensions of eigenfunc- 
tions, see Corollary 3.3 and Corollary |5.2| (i), that the connection coefficient formula 
is valid for all x G i. The connection coefficient formula holds then for all 7 G Sreg 
by continuity. 

(v) This follows from Lemma ^^(iii), part (iv) of the theorem, and Lemma lO] . □ 
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Remark 5.10. Observe that the statements of Theorem |5.9| (i) and (iii) are not in 
contradiction with the self-adjointness of {L,V), see Proposition |2.7| . Indeed, let 
7 e C \ R with I7I < 1, then 7 e S+g and /i(7) e C \ R. By Theorem U(i) and 
(iii) we have that E V and (^L^j){x) = fi{'-f)^^{x) for all x G {—l,oo{z))q, 
but the eigenvalue equation does not hold in the end-point x ~ —1. In fact, the 
self-adjointness of (L,!)) forces that (L<I>-y)(— 1) ^ /i(7)<I>^(— 1). Another proof of 
this inequality can be given using the non- vanishing of the Wronskian 11^ ((/)-,,, $7), 
cf. the proof of Corollary 5.2(iii). 



6. The Green function 

In this section we define the Green function of the self-adjoint operator {L,V). 
With the proper extensions of the asymptotic expansion $^ now at hand, see The- 
orem 5.9, the construction of the Green function is a straightforward extension of 
the construction given by Kakehi jll] and Kakehi, Masuda and Ueno jl^ for the 
little g-Jacobi function transform. We use some standard terminology and results 
for unbounded self-adjoint operators and their spectral measures for which we refer 
to Dunford and Schwartz Chapter XII] and Rudin Chapter 13]. 

Let (a, 6, c) G V^/"^" and /i e C \ M. Let 7 G C* be the unique non-zero complex 
number such that fi = ^(7) and ]7] < 1. Observe that 7 G C\M, hence 7 G Sreg aud 
^^(7) 7^ 0; see Theorem ^.9Kv). We define the Green kernel K^{x,y) for x,y E I 

by 

^ \W{jr'^^ix)%{y), y>x ^ ^ 



Ob serve that Kj{x, •), Ky{-, x) G P for all a; G / in view of Lemma and Theorem 
5.9 (i) and (iii), so we have a well defined linear map H mapping / G 7i to 



Gf{x,-/)^{f,K^ix,-)), xel. (6.2) 
Written out explicitly, we arrive at the formula 

(6.3) 

By the self-adjointness of (L, V) we have that the resolvent (L — /i.Id) "'^ is a one 
to one, continuous map from H onto T) for all /j, G C \ M. 

Proposition 6.1. Let (a, 6, c) G V3"\ f e H and fj. = ^(7) G C \ M with 7 G C* 
and \j\ < 1. Then Gf{-,j) = {L — fi.Id^ f. In particular, Gf{-,j) G V. 

Proof. Wc first prove that 

{{L~^4^))Gfi■,l))ix) = fix), yfen. (6.4) 

For the proof wc need to consider the two cases x G (— l,oo(z))q and x = — 1 
seperately. 

Case 1: x G (— l,oo(z))q. 
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Observe that the product rule (3.3) for the q-derivative Dq and Lemma 2.1 imply 
the following product rule for L: 

{L{fg)) (x) = (Lf) {x)g{x) + f{x) (Lg) (x) + qp{x)r{x){Dqf) (x) (Dqg) (x) 

+ p{x)r{q-'x) {Dqf) (q-'x) {Dqg) {q^'x). 

Combined with the easily verified formulas 

Dqfx^ I fiy)dqy] = f{x), Dqix^ [ f{y)dqy] = - fix) 



and the definition (6.3) of Gf{x,j)^ we obtain 

W-(7)((L-M7))G/(-,7)) {^) = 

<S>^{x)p{x) {Dq(p-\fcj)^)) (q'^x) - M^)p{x) (Dqip-'rf^-,)) iq~'x) 
+ p{x) {p-\f{Dq^^)c^^) (q-'x) ~p(x) {p-\f{DqC^^)<^>^) (q-^x) 
+ q {Tf{Dq<^^)4>^) {x) - q {rf{Dq^^)<i>^) (x) 



for f G H. By writing out the Dq-terms in this formula, we see that the right hand 
side reduces to (j).y){x)f{x), which in turn is equal to W{'-f)f{x) by Theorem 

p^(v). This completes the proof of (6.4) for x e (— l,oo(z))g. 
Case 2: x = — 1. 

Observe that the Green function in the end-point —1 reduces to 



G/(-l,7) 



/(y)<f7(y) 



dqy 
p{y) 



since 1) = 1. Using the product rule ( |3.3| ), we then have that 

Wij){DqGfi;j))i-l) = $^(-q)<^^(-l)Z|5il 

-C^^{^q)^^{-l)^-^+{DqC^^){-l)Gf{-l,^). 

Combined with the expression of L in the end-point a; = — 1 in terms of the q- 
derivative and the functions p{-) and r(-), see Lemma 2.1, and using the fact that 
(L(/)^)(— 1) = /i(7)(/)-y(— 1), we obtain 

W{j) {{L - A*(7))G/(., 7)) (-1) - W^(<i>^, 0^)(-l)/(-l). 



Hence (6.4) is valid for the end-point x = —1 since VF(<i>^, 1) — W(j) by 

Theorem 5.9 (v). 



It remains to show how (3.4) leads to a proof of the proposition. Let be 
the set of functions / : / C with finite support. Then Vfin C V C TC a.s 
dense subspaces. Let / G "Dfin, then Gf{-,^) is continuously differentiable at the 
origin since 0^, £ ^a'(7)- Furthermo re, G f{x,^) is a constant multiple of '^■yix) 



for a; > 0, hence G/(-,7) £ H. By (|6.4| ) it follows that LGf{-,-i) £ 7Y, hence 
Gf{-,^) £ v. Combined with ( |6.4| ), this proves that 

(6.5) 



{{L - ti{^)M)-^f){x) = G/(x,7) = {f.K^{x, .)), V/ £ P/. 



for all x £ /. By continuity, ( p.Sj ) is valid for all f £ H. This completes the proof 
of the proposition. □ 
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Remark 6.2. Proposition 6.1 is in general not valid when 0^ is replaced by ip^ and 



W{^) is replaced by the constant value of W{^-y, ipj) on / in the definition of the 
Green function G/(-, 7). Indeed, for the proof of ((L — fi{j))Gf{-, 7)) (—1) — /(—I) 
in the previous proposition, we use the fact that (p^ is a solution of (i/) (x) = 
/i(7)/(a;) in the end-point x = —1. This property fails in general to be true for ip-y, 
see Corollary 5.2 (iii). 



Proposition 6.1 plays a crucial role in determining the explicit form of the reso- 
lution of the identity L — J^tdE{t) for the self-adjoint operator {L,!)) on H since 
the spectral measure E is related to the resolvent of L by 

(i?((/il,M2))/,5) 

1 



lim lim 

SIO clO 2711 J^^^s 



where < 112 and f £ V, g G Ti, see p, Theorem XII. 2. 10]. In the following 



two sections we use Proposition 6.1 and (3.6) to give an explicit description of the 
continuous and discrete contributions to the spectral resolution E. 



7. The continuous spectrum 

We start this section by proving that the closed interval [—(1 + a)-^, —(1 — a)^] 
is contained in the continuous spectrum (Jc{L) of (L, T>). In section ^ we will see in 
fact that this interval is exactly equal to ac{L) for (a, 6,c) G V"/"^". Furthermore, 
we compute the spectral projection Pc := (1 + a)^, —(1 — a)^]) explicitly, and 
give the Plancherel formula and inversion formula for the continuous part of the 
big g-Jacobi function transform. 

For n £ N and x € I we set ip^\x) — (j)^\x)/\\cj)':y^^ \\, where (/i^"' G H is defined 

by 

['^7^ if X e I\[zq-''-\oo{z))g, 
'^^ ^""^ \0 if [zg-"-i,oo(z)),. 

Lemma 7.1. Let fi E [—(1 + a)^,— (1 — a)^] and 9 G [0,7r] such that /i = /i(e*^). 
Then (Vei'ejngpj ^■s generalized eigenfunction o/(L,P) with generalized eigenvalue 
IX. Furthermore, the continuous spectrum ac{L) of the self-adjoint operator 
contains the interval [—(1 -|- a)^, —(1 — a)^]. 

Proof. Recall that {^''fAa) ^^^^ij is a generalized eigenfunction of (L,!)) with general- 
ized eigenvalue ^(e'^) if (p^^} G V, = 1 and lim„_»oo (L</j^ri ~ ^i{e'-^)(p^^}) = 
in Ti. This can be checked for all 6 G [0,7r] by an elementary computation using 
Proposition |5.2| (ii) . 

Let /i G [— (1 -t- a)^, —(1 — a)^]. Then /i is part of the spectrum cr(L) of {L,!)) 
since there exists a generalized eigenfunction of {L,V) with generalized eigenvalue 
yLt. Then ^ is in the continuous spectrum ac{L) of (L, T)) or in the point spectrum 
o'p{L) of {L^V) since a self-adjoint operator does not have residual spectrum, see 
Theorem 13.27]. 

It remains to show that /i ^ (Jp{L). We have to distinguish between the cases 
PL G (-(1 + a)2, -(1 - af) and ^ = m(±1) = "(1 T a? ■ 
Case 1: n G (-(l + a)2,-(l - a)^), i.e. ^ = /i(e^^) with e G (0,7r). 
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It is a straightforward consequence of the asymptotic behaviour of <i>^ to infinity, 
see (O), that any non-zero hnear combination of the basis elements {$eie , $6-'" } 
of VZ~if,s^ does not he in M,,(pie-) (see Lemma |4.2| for the definition of M^), hence 



M ^ ap{L). 

Case 2: fi = fi{±l) = -(1 T a)^- 

Observe that ^^|-y=±i G ^^^±1) since |^(7)|-y=±i = 0. Using that the asymp- 
totics to infinity of is given by 

^(zg-™) =TO7-i(a7r(l + 0(9")), m oo, 

we obtain that 

M^(*±i,^l7=±i)(^) = -«^^0, xe/+. 



compare with the proof of Lemma [4.3[ Combined with Lemma 3.1(i), it foUows 
that {$±1, ^'^l7=±i} is a basis of V^^j-ij- It is easy to see, using asymptotics to 
infinity, that any non-zero hnear combination of these basis elements does not lie 
in M^(±i), hence ^^(±1) ^ cTpiL). □ 

Let Vfin C 7i be the linear subspace of functions f : I <C with finite support. 
Observe that Vfin C T> C Ti as dense subspaces. We define the big g-Jacobi 
function transform by 

(•^/)(7)-(/,07>- / /(x)07(x)-^, /eP/™, 7ee. (7.1) 
J-i P\X) 

In this section, we will regard the big g-Jacobi function transform J- f of / S 2?/™ as 
a function on the unit circle T. Observe that Tf is M^-invariant, where W = {±1} 
acts by {e.g){-i) = 5(7'), ^&W. 

We define an absolutely continuous measure on T by 

^^^^^ ^ dl ^ 1 d-j ^^2^ 

47riX c(7)c(7~-'-)7 4711X10(7)1^7 

where K is the positive constant defined by ( |2.14| ). Observe that the measure dv 
is a well defined measure on T since (a, 6, c) G V . In particular, possible zeros of 
the denominator of the weight function l/|c(-)|^ are compensated by zeros of the 
numerator. 

We first show that T extends uniquely to a partial isometry J- : TL ^ (T, diy^ , 
where (T, dzy) is the L^-space of T^-invariant functions with respect to di', when 
(a,6, c) € y/^". We start with the following crucial consequence of ( |6.6[ ) and 
Proposition |6?l| . 

Proposition 7.2. Let (a, 6, c) G V7™. Choose -(1 + a)^ < fii < < -(1 - a)^ 

and /ef < 6*2 < 6I1 < TT such that fij = ^(e^^^) {j = 1, 2). T/ien 

(i?((Mi,M2))/,5) = ^ J^^ {me''Hme'')j^ (7-3) 
/or a/? f,g e Vfin ■ 
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Proof. For £ C \ R we write j[fj] G C \ M for the unique complex number with 
modulus less than 1 such that fi{-^[fi\) — fi. Fix f,g ^ T^fin- By a straightforward 



computation using Proposition 6.1, we have for /i G R and e > that 

W{'y[^i±ie]) ^ 2 



'-jir (7.4) 

.{f{xm + fiy)W))^,^y 

p{x) p(y) 

where dx.y is the Kronecker-delta. Let —(1 + a)^ < /^i < A* < < ^ (1 — a)^ and 
let < 02 < 6* < ^1 < Ti" such that = ^(e'^) and fij = fi{9j) for j = 1, 2. Then we 
have 

lim7[/i±ie] = e=F^^. (7.5) 



Using the connection coefficients formula given in Theorem 5.9(iv) and using the 



fact that W{'^) ^ and ^^±i{x) {x G /) are regular at 7 G T \ {±1}, we obtain 

)*7[M-*e](y) \ 



lim 



^ <l>e-o{x)'^e-o{y) _ (l>e-<>{x)^^ie{y) 

1 (j)^w{x)(j)^w{y) 



It follows now by symmetrization of the double g-Jackson integral that 
hm (((L - {^l + ^e)y^ f,g) - {{L - (/i - ze))"' /,.g)) 



1 ff <t>e"^{x)(l),je{y){f{x)g{y) + f{y)g{x)){l-\5x^y) dqX dqV 



aK JJ |c(e'^)|2(e-»« - e^») p{x) p{y) 



{x,y)eiXl 

x<y 



1 (■F/)(e'^)(.Fg)(e'^) 
aK \c{e^<^)\-^{e~^s - e^'^)' 

The proposition follows now for all — (1 + a)^ < /i-i < /j,2 < — (1 — a)^ using ( |6.6| 



and changing the integration variable to 9 using the map 9 1-^ Ai(e ), see (|3.5|). The 
result now also holds when /ii = —(1 + a)^ or fi2 = —(1 — a)^ since the spectral 
measure E is countably additive. □ 

Corollary 7.3. The big q-Jacobi function transform J- uniquely extends to a con- 
tinuous linear mapping J-c'.Ti^ Ly^ (T, di'^ . 

If {a,b,c) G V^'^^ , then J-c : Ti ^ L^(T, di^) factorizes through the orthogonal 
projection Pc — E(^[^{l + a)'^,^{l — a)'^]) {i.e. J-c ^ J'c° Pc), and the restriction of 
J-c to the range TZ{Pc) of Pc is an isometric isomorphism onto the range TZ{!Fc) C 
Ll,{T,dv) ofT,. 



Proof. We assume first that (a, 5, c) G V^f^". In view of Proposition 7.3 applied to 
the special case fxi — m(^1) — ^(1 + o^Y and /12 = m(1) — ^(1 ^ a)'^, it suffices 
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to observe that i?({/^(±l)}) = 0, which is a consequence of Lemma 7.1 and [ po| 
Theorem 13.27]. 

Finally, observe that the inequahty H^/Hi < II /IP for / e C/in and {a,b,c) £ 
y^gen^ where ||.||2 is the norm of L^(T, dz/), holds for all {a,b,c) G F by conti- 
nuity. Hence J-' can be uniquely extended to a continuous linear map J-c : Ti. 
L^,(C*,di') for all parameters {a,b,c) □ 

Definition 7.4. We call Tc the continuous part of the big q-Jacobi function trans- 
form. 

Observe that the limit (/, (j>^)i = lim^.m^oo (/j 4>i)k;i,m exis ts for all 7 G T since 
<f>^ is continuously differentiable at the origin, see Lemma It follows that 

(•^c/)(7) = , lim (/, ^i)h 7 e T a.e. (7.6) 
for f ^ Ti^ where (., .)i (l G Z) is the truncated inner product 

if, 9)1= r f{x)W)^y (7.7) 
J-i P(x) 

In the remainder of this section we show that J-c is surjective and we give an ex- 
plicit formula for the inverse of the isometric isomorphism jFc : TZ{Pc) — > (T, dv) . 
The methods we employ are similar to the ones employed by Gotze Pj and by 
Braaksma and Meulenbeld [|| for the classical Jacobi function transform, and by 
Kakehi and Kakehi, Masuda and Ueno ||l2| for the little q- Jacobi function 
transform. 

Lemma 7.5. Let 7, J G T with 7^ l^{5). For k G Z+, l,m E Z with I < m, we 
have 

W^(0^, 05)(zg'-i) - Wi^^, c^5){zq^) + H^(</>7, <^5)(-g') 



Proof. For 7,5 G T we have that {L(j)^ )(j)s — 4>'y{L(j)s) = (a*(7) ^ t'-{^))4'i4's on 
/. The proof follows now from Lemma 2.2 since 0^ ; / ^ C is real valued for 



7 G T. □ 

We define a hnear map Qc : (T, dv) ^{I) by 

{Qcg){x)^ j g{-f)^^{x)dv{-t), xel. (7.8) 

Observe that 



{Qcgi,Gc92)kd,m ^ 9ih)g2{'-f'){(f>f,(j)Y)k-i,nidiy(j)di^{j') (7.9) 



T JT 



for k G Z+, ^ < TO in Z and 51,(72 G L^(T, d;/). 

Lemma 7.6. The limit {Gcgi,Qc92)i = ^'as-k,m^oc{Qc9iiGc92)k-i,ra exists for all 
91,92 G L^(T, di/), and 



{Gcgi, 0092)1 =11 9i{l)92{i){4''y,(t>i')idv{l)dv{j') 
Jt Jt 



51(7)52(7')— r-Tl . dv{-i)dv{-i ). 
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Proof. Observe that there exists a constant Ki > such that \{(l)j,4>-y')k:i.m\ < Ki 
for all 7,7' G T and for all k,m ^ Z+ with m > I. The first equality follows then 
from Lebesgue's dominated convergence theorem. 

The second equality follows from Lemma 7.5, using that M^(0.^,0y)(O+) = 
W(^(j)j,(j)j'){0~) since (j)^ is continuously differentiable at the origin, see Lemma 

□ 



3.5 



Finally, we determine the limit {Qcgi,Qc.g2) = lim/- 
is as follows. 



Proposition 7.7. The limit (Gcgi,Gc92) 
51 1 52 e i^(T, di^), and 



lim; 



-oo{Qcgi,Gcg2)i- The result 
AQcgi,Gcg2)i exists for all 



{Gcgi,Gcg2) 



gi{i)g2{i)dv{j). 



(7.10) 



H. is an isometric isomorphism onto the range 



In particular, Gc ■ L'wi'^T^'^) 

n{Gc) cn ofGc- 

Proof. We only sketch the proof, since it is similar to the little g- Jacobi case, see [|2[ 
Proposition 6.1] and |ll|, Proposition 7.4]. Let Cw(T) be the algebra of complex 
valued, continuous, M^-invariant functions on T. We fix gi,g2 G Ch'(T) such that 
{— 1, 1} is not in the support of gi and g2. It suffices to give a proof of (7.10) for 
such functions gi and 52- We start with the second expression of {Gig i, Gig2)i in 
Lemma 7.(:, and replace by its c- function expansion, see Theorem |3 . 9K iv) . Using 
the estimate 



sup 



i?fc(e±»«)-i?fe(e^ 



ai9 



= 0{k{qaf), 



for < (5 < 7r/2, where Rk{l) = ^-yizq^'') - (07)*' (cf. ([l4|)), which can easily 
be proved using the mean value theorem, we may replace in the expression of 
limi^-oo{Gcgi, Gcg2h the function ^^{zq^~^) by its asymptotic value (07)^"' at 00. 
Combined with ( ^.15| ) it follows now from the bounded convergence theorem that 



lim {Gcgi,Gcg2)i-r 



lim 



gi{e^')g2(e^'')s^(e,9')d9d9' 



in^K nT-^^ooJ^ |c(e'»)|2|c(e*«')|2(^(e»^') -M(e*'')) 
provided that the limit in the right hand side of the equality exists, with 

s^{e, 9') = c(e-^)c(e^««')e'('"~i'(^^+««')(e"« - e'«^'). 

£,€£{±1} 

Since 0(7) is continuous and non-zero on T \ {±1}, the Riemann-Lebesgue lemma 
implies that the contributions of the sums of s„j with ei^ > tend to zero in the 
limit, so s„ 



9, 9') may be replaced by 
t^(9,9') =c(e*«)c(e-*^')e*("-i)(^-*')(e** - e~ 
+ cie-'^0)cie^'^')e'^"'''^^''-'He-'' 



id' 



-Ac{e'%c{e-'^ ) sin 



sm 



) 

f {2m - l){9 - 9') 
V 2 



+ (c(e-^^)c(e'^') - c(e^^)c(e-^^'))e*('"-i)(«'-«)(e- 
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Applying the Riemann-Lebesgue lemma again and using 

Ai(e^^') - nie''^) = 2a{cos{9') ~ cos{d)) = 4a sin (^^-^) sin (^^-^) 



we arrive at 



lim (g,5i,ec52)i-™- lim — - / / ^lL^^^^^D^{9,9')d0d0' 
m-*oo m-*oc 47r^A Jq Jq c(e ^"jc^e/" ) 

provided that the limit in the right hand side of the equality exists, where Dm is 
the Dirichlct kernel, 



^ sin((2m-l)(0-0')/2) 



' ' sm{{9-d')/2) 



The result follows now from the well known L^-properties of the Dirichlet kernel. □ 
Recall the notation Pc for the orthogonal projection £'([—(1 + a)^, —(1 — a)^]). 

Proposition 7.8. (i) ^c\n{ga) '■ T^iGc) ^ L^,{T, diy^ is a surjective isometric 
isomorphism. Its inverse is given by Qc ■ L^(T, di') Ti-iGc)- 
(ii) We have UiGc) = TliPc) for (a, b, c) G F/™. 



Proof, (i) It follows from Corollary |7.3|, Lemma 7.6 and Proposition 7.7 that 



J^c{Gcf))h)gh)diy{j) = lim lim l {Gcf,cj)-t)k-Lmg["f)du{-f) 



= lim lim {Qcf ,Qcg)k-i,m ^ I f{l)g{l)dv{l) 

L — ^ — oo k.l — *cxD jy 

for all f,g& Ly^ (T, dv) . Hence Tc o Qc is the identity on (T, dv) and Tl{j-^ = 
i^(T, dv). Consequently, we have Qc o ^ck(e,) = Id7^(e,)■ 

(ii) Assume that (a,6,c) G 1//™. Let / G n{Qc). Observe that ||/|| = ||J^c/||2 
by the previous paragraph and by Proposition [7.7| , where ||.||2 is the norm of 
L'^{T,dv). By Corollary [tJ, this implies that / G 7^(Pc)■ 

Let / G TZ{Pc). We have seen that Tc(Tl{Qc)) — L^(T, rfi^), hence there exists 
a function g G TZ{Qc) such that Tcf — ^cg- Now f — g & 'R-iPc) by the previous 



paragraph, and J-c\n(Pa) is injective by Corollary 7.3, hence f = g (z Tl{Qc), as 
desired. 

□ 



8. The point spectrum 

In this section we determine the resolution of the identity £^ on K \ [— (1 + 
a)^, — (1 — a)^]. Observe that the kernel K.y{x, y) of the Green function is meromor- 
phic as function of 7 G D), where ID = {u G C | < \v\ < 1} is the punctured open 
unit disc in the complex plane. Since 4>j{x) and ^j{x) are analytic at 7 G D, see 
Theorem ^.91 (11) for we have that the poles of K^{x, y) coincide with the poles 
of VF(7)~^, which in turn coincide with the zeros of the map 7 ^ c(7~^) (7 G D), 



where c(-) is the c-function defined by (4.7). In particular, the poles of K^{x, y) in 
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7 g D are independent of x, y G / and are given by the set 

abcz 



1 



1 



S'=<^-T- I e e {a,5,c},fc e Z+ : -r- < 1 ^ U <^ - 



re 



abcz 
3+T 



< 1 



(8.1) 



Observe that these poles of K^{x, y) are simple since at most one of the parameters 
a, 6, c is > 1. Furthermore, observe that S C Sreg when (a, b, c) G F/"^". 

Lemma 8.1. The function (l)^^ (7 G S) is an eigenfunction of{L,'D) with eigenvalue 
fJ-ij). In particular, fi{S) is contained in the point spectrum <Jp{L) of [L^T)). 



Proof. Let 7 e 5. Then (j)^ e ^(7) by Lemma 3^. Furthermore, if 7 ^ Sreg, then 



c(7)$^ e M^(^), 



where the first equality follows from Proposition 4.4, Since 0(7) and <f>-y G are 
regular at 7 G S" n S'^eg, it follows that (^) is valid for all 7 G S". Hence cj)^ ^ V 
and L(j)j — fi('y)(j)j on / for all 7 G S", as desired. □ 



Remark 8.2. If (a, c) G V"/™, then C S'reg and </)7|7_|_ = 0(7)$^ by Theorem 
5^(iv), so formula (|8.2| ) extends for these parameter values to the whole q- interval 



Let 7 G 5. It follows from Lemma BT that the linear functional / 1-^ (•^/)(7) 
with / G Vfin is bounded, where J- is the big g-Jacobi function transform defined 
by (7.1), hence it uniquely extends to a continuous linear fmictional on Ti. It is 
given explicitly by 

(^/)(7) = (/>7)> /e^- (8-3) 
We define the weig ht diy{{j}) for 7 G and (a, 6, c) G F/™ by 

d.m) = iRes (^^(^)4_r)) = ^^Res^ (^^(7)0(7-)) ' ^^''^ 



where K is the positive constant defined by (2.14). Observe that the poles 7 G 
S of the meromorphic function 7 1— > (2(7)0(7^^)) are simple for parameters 
(a, 6, c) G y/^". We derive explicit expressions for the discrete weights c?z^({7}) by 
relating the continuous part of the Planchcrel measure with the weight function 
A{x) — A(a;; io, ti, t2, is) of the Askey- Wilson polynomials. 

Observe that 

■^c(7)c(7-i)7 {91,9/1; 1)^1 

where {e, /, 17} is an arbitrary permutation of {a, 5, c} (taking multiplicity into ac- 
count), and where M — M{a, b, c; z) is the positive constant 

1., n2 , , . (a5,ac;g)^6l(-a6z,-acz,-6cz) 
M = -(a5,ac;,)^^(-6cz,-&cz) = (f, ^^.^i-l/z) " ^'-'^ 
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For a simple pole eq^ of A(x; ^i; ^2, ^3), where e £ {tj}^j=o and k £ Z+, we have 
the explicit formula 



Res 

7— eq'^' 



(9, e/, //e, eg, g/e, eh, h/e; q)^ 
{e^,ef,eg,eh;q)^ (1 - e^g^fe) / q 



' iq,qe/f,qe/g,qe/h;q)^ {I - e^) \efgh, 

see 10, (7.5.22)], where {f,g,h} is such that {e,/, = {^01*11*2,^3} (taking 
multiplicity into account). It is well known that the right hand side of ( ^.8D is well 
defined and positive for real parameters ti such that the titj (i ^ j) arc strictly less 
than one. 

Combined with (Q, (|||) and ( pl^ ), we obtain for e € {a, 6 c} and fc G Z_|_ 
such that l/q'^e < 1, 



(9, e/, //e, eg, 5/e; q)^9{-fgz, -e^fgz) 
{e\ef,eg-q)^ (1 - eV'=) /-g^'^+i^/^^ 



/5 



(8.9) 



' {q,qe/f,qe/g;q)i^ (1 - e2) 

where /, g are such that {e,f,g} — {a,5, c} (taking multiplicity into account). 
By the positivity of the residues of the Askey- Wilson measure, it follows that the 
right hand side of (8.9) is well defined, regular and strictly positive for parameters 
(a,5, c) e V . 

Similarly, we obtain for k G 'Z such that abcz/q^^^ < 1, 
diy{{-abcz/q''+^}) = 

M 



(g, q, —q/abz, —q/acz, —q/bcz, —a'^bcz/q, —ab'^cz/q, —abc^z/q; q)^ 
(y—q/abz, —q/acz, —q/bcz; g)^. 



(8.10) 



,(fe+3)/2 



a^b^c 



(— l/a^6c, —1/ab'^c, —l/abc^; q) ^ 

The right hand side of ( ^.10 ) is well defined, regular and strictly positive for pa- 
rameters (a, 6, c) e V . We will sometimes abuse notation by writing £^({7}) for the 
discrete weight ^(^({7}) (7 € S). 

Proposition 8.3. Let {a,b,c) G l^"™. 

(i) For /J,i < /i2 < — (1+a)^ or — (1— a)^ < /ii < ^2 such that /i(S')n(/ii, /i2) — 0, 
we have E (^{fxi , fi2)) = 0. 

(ii) For 7 G S' and /, g G H iwe have 



{E{Mj)})f,g) = (.F/)(7)(.F.g)(7)dK{7})- 
Proof. Throughout the proof, we use the notations introduced in the proof of Propo- 



sition 7.2 



(i) Let /, (? G Vfin and fix < ^2 satisfying the properties as stated in (i). It 
suffices to prove that (i?((/ii, /i2))/, .g) = 0. Observe that (7.4) is still valid in the 
present setting, but that the analogue of (7.5) is now given by 



lim7[/i ± ie] — 7, 



.11) 
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where < /i < /i2 and where 7 e D is the unique element satisfying /i — fi{j). 
By the condition n (/ii, /i2) = 0, we have for 7 e D satisfying /ii < < ^2 
that 

i™<(.-(M,)±..))-/.,)^ // 

'''^1%'" (8.12) 



By the bounded convergence theorem we may interchange the hmit e | and the 



integration in ( |6.6D . Combined with (B.12), this gives {E(^{^i, 112)) f, g) = 0, as 
desired. 

(ii) Let 7 G S* and f,g E 2?/in- Choose arbitrary /ii < /i2 such that /i(S') n 
(^i,/i2) = /^(7) and such that [^i, fi2] n [—(1 + a)^,— (1 — a)^] = 0. Then by 
the first part of the proposition, we have E({^{^)}) = E({^i, ^2)) ■ We compute 
now {E(^{fj,i, fj.2)) f, g) using (|6^ . We substitute (0) in (pl[ ) and change the 
integration parameter fi in ( |6.6[ ) to the 7-parameter using ( |3.5| ). Observe that for 
^1 < /i < ^2 and e > 0, we have that 7[/i + ie] (respectively 7[/x — ie]) lies in the 
lower (respectively upper) half plane of C. Since 1^(7)"^ has a simple pole in 7, it 
follows by Cauchy's Theorem that 



1 



{E{{pi,fi2))f,g)^ // {f{x)giy) + fiy)gix)){l^-d^ 



.Ux)^4y) f "(r' -7)Res(W^(7)-^; 



dqX dqy 



p{x) p{y) ' 

where the factor ^(7^^ — 7) arises from changing the integration variable in ( |6.6D 
to 7 using the map 7 1-^ /^(t), and from the fact that one has to change sign in 
order to get a positive oriented curve around 7. The proof is now completed using 



the explicit expression of ^^(7), using Remark B.2 and by symmetrizing the double 



g-Jackson integral. □ 

Corollary 8.4. Let {a,b,c) G V^"^ . The spectrum a (L) of the self-adjoint operator 
(L, I?) is given by 

a{L) = [-(1 + a)2, -(1 - af] U fi{S), 

where ac{L) = [—(1 + a)^, —(1 — a)^] is the continuous spectrum and <Jp{L) — ^{S) 
is the point spectrum. 



Pro of. It follows frornJEO], Theorem 13.27], Proposition 7^, Proposition 7^, Lemma 
|j] and Proposition |8.3| that a{L) = [-(1 + a)^, -(1 - af] U iJ,{S). Observe now 



that [—(1 + a)^, —(1 — a)^] C crc(L) by Corollary iTjJ and n{S) C crp{L) by Lemma 
|j[ It follows that CTc(i) = [—(1 + a)^, -(1 — a)^] and ap{L) — fi{S), since a{L) is 
the disjoint union of cf^{L) and a-p{L), see Theorem 13.27]. □ 

Corollary 8.5. The functions </>.y G P C ?i (7 G 5) are mutually orthogonal in Ti. 
Their quadratic norms are given by H^-yP ~ '^({t}) ^ ■ 
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Proof. Orthogonality is clear since the functions (7 G S) are eigenfunctions of 
the self-adjoint operator {L^V) with mutually different eigenvalues /^(7) (7 G S*), 
see Lemma 8.1. 

It remains to derive the explicit expression for the quadratic norm We 
first assume that (a,5,c) e V7™. Observe that E{{^jL{^)})(j)=^ = z/({7}) ||(/'-y|| V-y 
for 7 e S* by Proposition |8j(ii). Since ^ and E{{^i{^)}) is a 

projection, it follows that H^^IP = '^({7}) ^ for 7 G 5, as desired. The result now 
follows for (a, b,c) eV by continuity. □ 

9. The Plancherel formula, inversion formula and the dual 
orthogonality relations 

The explicit knowledge of the resolution of the identity E of the self-adjoint oper- 
ator (L, V) on Ti leads directly to the Plancherel formula and the inversion formula 
for the big q-Jacobi function transform, which we formulate in this section explicitly. 
We show that the dual orthogonality relations imply orthogonality relations for the 
continuous dual q^^-Hahn polynomials with respect to a one-parameter family of 
non-extremal weight functions. Furthermore, the dual orthogonality relations give 
explicit sets of functions which complete the continuous dual ^"^-Hahn polynomials 
to orthogonal bases of the corresponding L^-space. 

We define the measure dv{-) = dv{-] a, 6, c; z) on C* by 



f{i)dv{^) - I f{i)dv{^) + fm^m), (9.1) 



T 



where the measure dv{-^) on T is defined by (7.2), 5 C B is the discrete set defined 
by (|8.1| ) and the point mass dv{{'^}) for 7 G S* is defined by (8.9) for 7 > and 
(p.iq) for 7 < 0. 



The measure dv{-) on C* is well defined for (a, 6, c) G V . Indeed, for the abso- 
lutely continuous part of the measure the conditions on the parameters are such 
that the possible zeros of the denominator of the corresponding weight function 
l/|c(-)|^ are compensated by zeros of the numerator. For the discrete part of the 



measure it follows from the explicit expressions ( pT9| ) and (3.10) that dv{{'^}) is 
well defined and strictly positive for all 7 G 5. 

Let L'^(C*,dv) be the Hilbert space of VF-invariant L^-functions with respect 
to the measure dv. We define the big q-Jacobi function transform for functions 
/ G H such that f{zq-^) = for fc > by 

9{i)--={^f){i) = {L^,) = j ^ f{x)M^)^y 7ee, (9.2) 

and we define for functions g G L^(C*, dv^ satisfying g{~q''abcz) = for A: ^ 0, 
f{x):^{gg){x)= f g(7)0y(x)di.(7), x e I. (9.3) 



The results in section ^ and section || lead to the following main theorem of this 
paper. 

Theorem 9.1 (The big g-Jacobi function transform). Let z > and (a, 6, c) G V. 

The maps T and Q uniquely extend to surjective isometric isomorphisms 
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Furthermore, Q — J- ^, hence ( P^ ) and ( |9.3[ ) give the big q-Jacobi function trans- 
form pair (interpreted in the suitable -sense). 

Proof. We write (., .) for the inner product of L^(C*, di/). 



Suppose that (a,6, c) G V/'^" and let f,g € X^/m. Applying Proposition 7.2 and 
Proposition |8.3| , we obtain (/, g) = {E{R)f,g) — {Tf.Tg). In order to extend 
this result to parameters (a,5, c) in V, we show that (/, 5) and (j-f,J-g^ depend 
continuously on {a,b,c) £ V. This is clear for {f,g), while for [Tf,J-g^ it is clear 
except for the term 



E (-^Z) (7) (-^5) (7)^^(7) = E(^/)(-g'^a5cz)(^.g)(-g'=a6cz)d^.(-g'=a6cz), 

7eS:7<0 fcGZ 

where we use the convention that dv{—q^abcz) = if —q'^abcz S for the right 
hand side. The continuity of this term follows by Lebesgue's dominated convergence 
theorem, using the asymptotics 

<l>-q>'abcz{x) ^ 0{x-''), dv{-q''abcz) ^Oiz'^q''^''-^^'^), k ^ 00, (9.4) 

where x € I, which hold uniformly for (a, b, c) in compacta of V . To compute the 



asymptotics (3^) for 4'-y{x) with x G /+ we used (p.2|), and for a; G /_ we used the 
formula 



(q/ab, q/ac, qj'^ - 
{qi/a,q7/b,qj/c; q) 



M^) = ^;(^). (9-5) 



see Lemma 4.5 (observe that the right hand side of ( |9.5[ ) is well defined for 7 < 
with I7I < and can be uniquely extended by continuity to (a, b, c) G V"). 

It follows that {f,g) = {J^f,J^g) for f,g & 'Dfm and {a,b,c) G V, hence f 
uniquely extends to an isometric isomorphism J- :Ti ~^ L^(C*, dv) onto its image 
for all (a, 6, c) G V . 

Let / and g be M^-invariant continuous functions on C* with compact support. 
We have {gf){x) = {Qcf){x) + {Qdf)[x) for x G / with given by (0) and 

(erf/)(2^) = E/(^)'^7(^)rfK{7}), 

=tes 

which is a finite sum by the assumptions on /. Assume that (a, 6, c) G V^/*^" , then 
it follows from Proposition that Q^f G TZ{Pc). Furthermore, it follows from 

the proof of Corollary U that Gdf G 7^(Pd) = (7^(Pc))^, where Prf = E{^i{S)). 
Hence Gf eH and {Qf,Qg) = {Qcf,Gcg) + {Qdf,Qdg)- It follows from Proposition 
O and Corollary Is^ that 



{Gcf^Gcg)^ [ 7(7)5(7)^^(7), (ed/,ed5) = E/(^)i?(^)'^^(W)- (9.6) 

This impHes that {Gf,Gg) = (f, g) ■ 

It follows from Proposition |7.8| (i) and Corollary that (^^) is valid for all 
(a, 5, c) G V. Furthermore, by continuity arguments, we have {Gcf, Gdg) = 
for all (a, b, c) G V. Hence, G uniquely extends to an isometric isomorphism 
G ■ L^(C*, (ii/) H onto its image for all (a, b, c) G V. 

A direct computation now shows that 

i^f, 9) = if, Gg), yfen,Vge L^r (C* , dv) 



THE BIG ij-JACOBI FUNCTION TRANSFORM 35 



for (a, 6, c) G V, cf. the proof of Proposition 7.8. This imphes that = 
L^(C*, di^y, Tl{G) = H and Q = J-^^, which completes the proof of the theorem. 

□ 

In the remainder of this section we derive a one-parameter family of non-extremal 
orthogonality measures for the continuous dual g^^-Hahn polynomials, and explicit 
sets of functions which complement the polynomials to orthogonal bases of the 
corresponding Hilbert spaces. 

We write io = — ^/b, ^2 — 1/c- The condition (a, 6, c) G 1/ is then 

equivalent to the conditions 

U > 0, Ut, > 1 ^ j) (9.7) 

on the parameters (io:^i:^2)- We define polynomials Pk{'^) — Pkij',to,ti,t2', 
{k G Z+) in 7-^7"^ by 

(9.8) 

Here we use Exercise 1.4(i)] to obtain the second equality. Observe that the 
second equality of (9.8) shows that {pfc(-; io, ^i, ^2; '?^^)}feez+ are exactly the con- 
tinuous dual g^^-Hahn polynomials, i.e. Askey-Wilson polynomials in base 
with one of the four parameters equal to zero. For z > 0, we define a measure 
dcr2(.) = dCT^(.;to,ti,t2;'7"^) on C* by 

f{l)daA,l) = ^ J^Jh)d,y{r,to\t^\q';z). (9.9) 

Explicitly, we have 

/(7)da,(7) - / fh)wAl)- + E /(^)Re^ ' (9-10) 

47^^ Jj 7 7=7 V 7 / 



where 



k k 

S,^ S-^ ={^\ee {to, h,t2}, fc G Z+ : ^ > 1 
e e 

-q''tQtit2 I ^''^0*1*2 ^ 

\k E £ : > 1 

z z 

and with weight function Wz{-) = Wz{-',to,ti,t2', q^^) given by 

Ml) = -, V 2°° . ^ (9-11) 

6[-z-imtit2, -z/totit2"f) Uj=ou/tj^ lAj7;9)oo 

for parameters (^o, ii, ^2) such that the poles of 102(7) at 7 G S'z are simple. Finally, 
we define W-invariant functions r^{'j) — r^{j;tQ,ti,t2', q~^) for A: G Z by 

rUl) = ^^izq'';to\t^\t2') 

_ {l/taA/tit2,-q''z/totit2T,q)^ , / 1/^17, 1/^27, -g''2Ait2 



{l/toh,l/tot2,~q'^z/ht2;q)^''^' { l/ht2,-q''z/toht2^ ' 

(9.12 



where the second equality holds for 7 G C* with |7/<ol < 1, see (3.8). 
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Theorem 9.2. Let z > and fix parameters ti satisfying the conditions (3.7). 
Then, {pk}kez+ ^{fk]kez is an orthogonal basis of the Hilbert space (C* , (itrT) ■ 
The quadratic norms of the basis elements are given by 



\Pk{l)\ da,{"f) 



l^-z/toti, -z/tot2, -z/tit2) 



1 (9,1/^2; 9);, . 

y 5 



■ (q, i/Vi, iAoi2, 1/^1^2; g)^ (lAo^i, iAoi2; q)^. 

0{-l/z) 



C* d{-z/tQti,~z/tot2,-z/tit2) 



1 {-q''z/toti,~q''z/tot2;q)^ 



-2°q-'' 

(lAoti, lAot2; <Z)1 {-q''z/tit2, -q^+^z- g)^ 



Proof. It follows from Theorem |9.1| that 



{7 h-> (/..^(-g'^) I fc e z+} u {7 I fc e z} 



is an orthogonal basis of (C* , dv) , and that their quadratic norms are given by 

(1 - q)q'' 
f) 

(1 - 

The theorem follows now immediately by setting to — l/oi — ^2 = 

1/c and using the explicit expressions (2^) and ( ^.7| ) of the function p(-) and the 
constant M . □ 



|07(zg'=)|2dj.(7) = TT^^^V^, A; G Z. 

1 - (7 ZO*^ 



Remark 9.3. We remarked in section |l| that the big q-Jacobi function transform 
is associated with harmonic analysis on the SU{1, 1) quantum group. Analogous 
considerations for the quantum group of plane motions lead to the so-called big 
q-Hankel transform, see [p^ and The corresponding function theoretic aspects 
of the big g-Hankel transform are discussed in detail in Q. 

The dual orthogonality relations for the big g-Hankel transform have a similar 
interpretation as the dual orthogonality relations for the big Jacobi function trans- 
form, namely, they give orthogonality relations for Moak's g-Laguerre polynomials 
with respect to a one-parameter family of non-extremal orthogonality measures, as 
well as explicit sets of functions which complement the g-Laguerre polynomials to 
orthogonal bases of the associated Hilbert spaces, see Theorem 4.1]. 

10. Big g-jACOBI POLYNOMIALS: FUNCTIONAL ANALYTIC APPROACH 

We show in this section how the orthogonality relations and the quadratic norm 
evaluations for the big g-Jacobi polynomials can be derived from a functional an- 
alytic approach. The arguments are closely related to the ones used for the big 
g-Jacobi function transform, so we merely sketch the steps and indicate the main 
differences. To avoid confusion with previous notations, we label definitions in this 
section with a subscript (or superscript) p (indicating that it is connected with the 
polynomial case). 
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The conditions on the parameters a, 6, c are now taken to be 

ab,qa/b,ac,qa/c < 1, 6c < 0. (10-1) 

We consider the second order q-difTerence equation L acting on the space J'{Ip) 
of complex-valued functions f : Ip C, where Ip = [—1, —q/bc]q — {—q'^}kez+ U 
{— g'^"'"^/5c}fcgz+- At the end-points x = —1 and x = —q/bc this should be read as 
(^Lf){x) — A{x){f{q x) — f{x)). We can write this in self-adjoint form, similarly as 
was done in Lemma |2.l| for the non-compact case, with the same functions p and 



r, see ( |2.6| ). Observe that p{x) > for all a; £ /p by the conditions (10.1) on the 
parameters. 

We define Hp ^ {f : Ip ^ C | ||/||2 = (/, f)^, < ^}, where 

(/,5)p= / f{x)g{x)^ 

I t is well known that the big g-Jacobi polynomials, which are explicitly given by 
form an orthogonal basis of Tip, see 0. This fact and the evaluation of 
the corresponding quadratic norms have been derived in ^ using the (/-binomial 
formula 0, (1.3.2)] and the (/-Pfaff-Saalschiitz formula [0, (1.7.2)]. In this section 
we derive these results by functional analytic methods. 

Similarly as in the non-compact setting, we truncate the inner product by 

/ +/ /(-)5(^)^^ 



.,1 + 2 



/be. 



p{x) 



for fc, Z £ Z+. The analogue of Lemma ^.2| is then given by 

{Lf,g)p^kj - {f,Lg)p,k.i = W{f,g){-q'') - W{f,g)(-q'+' /be), 



with the Wronskian as defined in (2.11 



Let a £ T, then we write Dp^a for the functions / £ Hp such that Lf £ Tip 
and /(0+) - a/(0-), /'(0+) - af'{Q-), where now /(O") = linife^,, /(-g*^') and 
/(O"*") = lim/j^oo f{^1^ /be), etc. (cf. section |^). By similar arguments as in section 
H, we have 

Proposition 10.1. The operator (L,!?^,^) on Tip is self-adjoint for all a £ T. 
We denote Dp = Now observe that 4>j,ip^ £ Vp for 7 £ C* by Lemma 



3.5 (see (3.6) and (3.7) for the definition of (p-y and ■07, respectively). Furthermore, 
from the arguments as given in section |^ it follows that (^L(j)^){x) = /x(7)(/)^(a;) for 
X £ [-1, -q/be)q = Ip\{-q/bc} and (Xy;^) (x) = fi{^)->jjj{x) for x £ (-1, -q/bc]q 



Ip \ {-1}, where fj.^j) is given by (|3.5|). The Wronskian Wp{"f) = ^^{ip^jfj)^) G 
J-{Ip) can again be seen to be constant on Ip (cf. Lemma 3.1(iii)), and an explicit 
expression of the Wronskian Vl^p(7) is given by 

{ai,a/-i\q)^e{be) 



Wp{^)^{l-q)- 



ab, ac, qa/b, qa/c; q) 



Indeed, observe that Proposition is also valid for the present choice (10.1) of 
parameter values by analytic continuation. 

In particular, the functions (jj-y and 1/^7 in I^p ^I'e linearly independent if and 
only if 7 ^ Spoi, where Spoi = {ln^}n£Z+, 7« = a?"- Hence {L<j)^){-q/bc) ^ 
Kl)4'7i-Q/bc) and (L'(/)^)(-l) ^ ^(7)?/;^(-l) if 7 ^ Spoi, cf. Corollary 



38 



ERIK KOELINK AND JASPER V. STOKMAN 



Let fj, = e C\M.. We define the Green kernel K!^{x, y) for y G Ip by 

''^^ \Wp{^)-^<j>,{x)i>,{y), y>x. 
We have a well defined linear map Tip — > J'ilp) which maps / G Tip to 



G^^ix,j)^{f,K!;ix,-))p, xelp. (10.2) 
By similar arguments as in the proof of Proposition 6.1, we derive that for / G Hp 



and for 7 G C* such that fi{-/) G C \ M, 

G^(.,7) = (i-M7).Id)"V. (10.3) 
For the proof of ( |10.3[ ), we have to consider case 2 of the proof of Proposition 



S.l twice, namely for the end-point —1 as well as for the end-point —q/bc. The 
arguments go through for both end-points, since (j)^ (respectively V7) satisfies the 
eigenvalue equation {Lf)(x) = /i(7)/(a;) in the end-point x — ~1 (respectively 
x = —q/bc). 

We are now in a position to compute the resolution of the identity Ep for the 
self-adjoint operator (L, Pp) on Tip using ( |6.6| ). For the moment it is convenient to 
assume that the parameters also satisfy the condition ^ {<l^}kei.- This condition 
can be removed later on by continuity. 

We keep the notations of the proof of Proposition 7.2. Choose f,g £ Hp with 
finite support. Then we have for /i G M and e > 0, that 



x<y 



(10.4) 



■i.f{^)9iy) + f{y)9M} 



dqX dqy 
p{x) p{y)' 



Let C > such that n{Spoi) r\[-{l + a)^ - -{1 - a)^ + (\ = 0. Using (|7J), (3.11 
and the invariance of ip^, tp^y a-nd Wp{'y) under 7 ^ 7^^, we obtain from (10.4) 
that for aU -(1 + a)^ - ^ < ^ < -(1 - a)^ + ^, 

hm (((i + ie))-'f,g)p - {{L - (/i - *e))~V,.9)p) = 0. 

It follows now from (jej) that Ep{[-{1 + a)^, -(1 - a)^]) = 0. 

For —00 < /ii < /i2 < —(1 + a)^ or —(1 — a)^ < /ii < yU2 < 00 such that 
(/^ii/^2) n fJ.{Spoi) ~ 0, we have _Ep((/ii, /i2)) — 0, cf. the proof of Proposition 
^■3| (i). Setting /i„ ~ l^iln) for n G Z+, we have, due to the simple pole of Wp{'^)~^ 
in 7 = 7„, 



{Ep{M)f,g)p^ II {f{x)giy) + fiy)g{x)){l--Sx^y) 



x<y 



4,Jx)^,,Sy) i-hn'-in) Res {Wp{^r')) 
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for J,g ^ Hp with finite support, cf. the proof of Proposition Using ( ^.7[ ) 

to rewrite V'7„ as a multiple of , and by symmetrizing the double q- Jackson 
integral, we obtain 

(-Ep({Ain})/,5)p = A/'p(")"^(/,V'7Jp(^7.:5)p> (10-5) 

where 

Kin) . f - ,J i^^^l^^Zfpk Re, " 



(l-g)(q,6c,g/6c,g2"+ia 


^9 




[q,q''a^;q] 


n 


{qa/b, qa / c, g"a6, g"ac; g 


oo( 


ga/6, qa/c; g) 


n 



q(3-n)/2 

Vc 



Observe that ( 10. 5| ) holds for all f,g £ Tip by continuity. 



We can now immediately recover the orthogonality relations and quadratic norm 
evaluations for the big q-Jacobi polynomials by applying well known properties of 
resolution of identities to Ea. The result is as follows. 



Theorem 10.2 If the parameters {a,b,c) satisfy the conditions (10.1), then 

the polynomials {^-y^^n^^ form an orthogonal basis of Hp, and their quadratic 
norms are given by 
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